How to increase google crawl speed for your website

google_crawl_faster

A question that often pops up for sites that are chasing “hot topics” is: how to get Google to crawl our site faster? This could be either because you have a big site and want to see changes faster or because you’re optimizing a news site. There are a few things you can do to improve your crawl speed. We’ll cover them in this post.

There are a couple of possible reasons why Google is slow when spidering your site. The first might seem obvious: if Google doesn’t find enough (quality) links pointing to your site, it doesn’t think your site is very important. The other reasons are technical: it has too much to crawl on your site, your site is too slow, or it’s encountering too many errors.

Your site doesn’t have enough authority

When your site doesn’t have enough quality inbound links, Google will not crawl your site very quickly. You can imagine it doesn’t necessarily want to spend much time on a site that it doesn’t deem important. When your site is brand new, this will be the case.

So, when you need more crawl action on your site, start doing some link building.

Technical reasons for slow crawling

The technical reasons for Google to crawl your site slowly can be divided into three groups: your site is too slow, you have too many errors, or you have too many URLs.

Your server is slow

The main reason why we see Google crawling sites slowly, is when the site itself is actually slow. Google actually indicates this to you when this is the case, on the “crawl rate” page of Google Search Console. If this is the case, Google might show you a “faster” option:

Crawl faster

Our advice: set it to “Faster”, but don’t stop with that: make your server respond faster, either by upgrading your hosting, or improving your site’s caching. Chances are that when Google is suffering long load times, your users are as well.

Too many errors on your site

If you have a lot of errors on your site for Google, Google will start crawling slowly too. To speed up the crawl process, fix those errors. Simply 301 redirect those erroring pages to proper URLs on your site.

If you don’t know where to find those errors: use the Google Search Console integration in Yoast SEO. Or log into Google Search Console. If you have access to your site’s access logs, you can also look at those, preferably with a tool like Screaming Frog’s Log file analyzer.

To prevent your site from being crawled slowly, it’s important that you regularly look at your site’s errors and fix them. We have a more extensive article on fixing 404 errors.

When your site is down too much, Google will slow down incredibly hard. If this happens because of your hosting, switch hosting. Make sure to check on your site’s uptime with a tool like Pingdom.

Too many URLs

If you simply have too many URLs on your site, Google might crawl a lot but it will never be enough. This can happen because of faceted search navigation for instance, or another system on your site that simply generates too many URLs. To figure out whether this is the case for you, it’s always wise to regularly crawl your own site. You can either do that manually with Screaming Frog’s SEO spider, or with a tool like OnPage.org.

Tips to increase your crawl speed

A couple of simple tips to increase your site’s crawl speed:

  • Using the methods above, find and fix all the errors.
  • Make sure your site is fast.
  • Add an XML sitemap to your site and submit it to the search engines.
  • If all of that fails to improve your crawl speed, start link building!

How to change location in Google search result?

map-local-search

After guide of Penguin 4.0

penguin-4

Penguin 4.0 was announced on September 23, few points of it:

  • The algorithm now devalues links rather than punishing sites.
  • Penguin is baked into Google’s core algorithm, updating in real time.
  • The feasibility of negative SEO is greatly diminished.
  • The new Penguin is more granular.
  • Penguin 4.0 pushes SEO closer to real marketing.

Penguin 4.0 is the relief many sites have waited over two years for.

The algorithm now devalues links rather than punishing sites

This new version of Penguin devalues spam links rather than penalizing entire sites.

When Penguin first launched, it was punitive by nature. If your site was affected by Penguin, your entire site was demoted in search — without clear explanation or instruction as to how to recover. The harshness of Penguin, combined with the misinformation surrounding the algorithm, created a negative environment and contentious relationships between business owners and Google.

Penguin 4.0 — and its devalue vs. demotion approach — is a vast upgrade, and it should improve relations between site owners and Google.

Google will still be able to prevent spam from manipulating their results, but now site owners won’t have to live in fear of a crippling Google penalty, with questionable recovery. Manual actions still exist, but these are made by discerning humans who are much better at determining the intent behind links. Manual penalties are also much easier to identify, both in application and in recovery.

Penguin is baked into Google’s core algorithm, updating in real time

Penguin 4.0 is now baked into Google’s core algorithm, and updates in real time.

With this change, Penguin’s data is refreshed in real time, so changes will be visible much faster, typically taking effect shortly after we recrawl and reindex a page.

This means no more waiting for Penguin refreshes or updates in order to recover. (Sites demoted by Penguin waited over 700 days for this latest update!) In fact, Google stated in their announcement that they won’t comment on any future refreshes.

This is great news, and it should relieve more of the fear, misinformation and uncertainty surrounding Penguin. Now that Penguin 4.0 is real time, site owners are no longer at the mercy of Google’s Penguin refreshes, which happened infrequently at best. This is particularly beneficial to SEOs, as competitors outranking our clients via spam will now be devalued in real time.

The feasibility of negative SEO (NSEO) is greatly diminished

Penguin 4.0 should help end negative SEO problems.

Either way, Penguin 4.0 should alleviate many concerns over negative SEO.

The goal of NSEO was to have your competitor penalized, which is no longer possible, since Penguin won’t demote entire sites.

Of course, manual actions are still possible. But according to Illyes, there isn’t any “trigger” you could trip with negative SEO that would flag a site for manual review.

The new Penguin is more granular

This new iteration of Penguin is much more granular than previous versions.

Penguin now devalues spam by adjusting ranking based on spam signals, rather than affecting ranking of the whole site.

This is another significant improvement for Penguin. Rather than devaluing an entire site, Penguin can now pinpoint spammy links and devalue them at the page level.

Penguin 4.0 pushes SEO closer to real marketing

The improvements made to Penguin 4.0 provide more immediate rewards for legitimate practices because spam is devalued more efficiently. This shift will push the SEO industry as a whole toward real strategies, and away from manipulation.

Of course, many successful and respected SEOs are already involved in real marketing. However, there are also many shady individuals and companies that claim to be SEOs while providing spammy services.

Penguin 4.0 is another nail in the coffin for these unscrupulous agencies, and that’s good for the SEO industry as whole.

Sites propped up by spam will be devalued, providing gains for those who secure links the right way.

Complete Basic Guide of Google Ranking Signals: Part 8

Group Holding Together

Last week we published the seventh instalment of our complete guide to Google ranking factors.

It concentrated on site-level signals, such as HTTPS, speed, mobile friendliness and structured mark-up.

This week, we’re looking at internal links (before we tackle the much more unwieldy subjects of outbound links and backlinks).

Internal linking

What’s an internal link? Well if you click on that link, you’ve just discovered one for yourself.

The practice of internal linking has many advantages, that can help your site improve its metrics and the user experience.

1) Internal links can help navigate people around your site in a more targeted fashion.

2) Internal links can keep people on your site, particularly if the links are relevant to that particular webpage.

3) They provides your audience with further reading options, and if they continue to click around your site without leaving, this can help reduce your bounce rate (the percentage of people who left a given page on your website without viewing any other pages.)

4) Internal links help Google crawl and index your site. The Googlebots that are sent out to fetch new information on your site will have a better idea of how useful and trustworthy your content is, the more they crawl your internal links.

5) Search engines will see that some of your webpages have more internal links pointing towards them than others, and will therefore judge them as more important.

6) The higher the authority of a page on your website, the more valuable its internal link becomes.

7) According to Starcom’s Jason McGovern, internal linking is one of the few methods we can use to tell Google (and visitors) that a particular page of content is important.

From a strategic perspective, it helps webmasters bridge the ‘authority gap’ between their most linkworthy content and their most profitable content.

For instance you can use a link from an evergreen post with lots of search visibility and traffic to promote something relevant your business needs to raise awareness of.

8) Broken links send a bad trust signal to Google, as it makes your site look incompetent or irrelevant at best, poorly maintained or abandoned at worst.

Anchor text

9) By using clear anchor text (the clickable highlighted words in any give link) it helps improve your ranking for certain keywords. If we want this article to rank for the term ’internal link guide’ then we can begin linking to it from other posts using variations of similar anchor text.

This tells Google that this post is relevant to people searching for ‘internal link guides’.

10) Some SEOs recommend varying the anchor text pointing a particular page as Google may see multiple identical uses as ‘suspicious’.

Hub pages

11) You may find that linking to a single hub page will help your site avoid cannibalising itself for search positions.

A hub page is a page themed around a certain topic or keyword.This page is constantly updated with fresh content, and is therefore always considered ‘fresh’ and valuable by Google.

Google is to add sitelinks icons into search result

Sergey Alakov posted screen shots on Twitter of Google testing a new look for sitelinks in the mobile search results.

These sitelinks now contain an icon on the left of the link, in the snippet. The icons can be buildings, person, briefcase, login, contact, map icon and more.

Here is one of the many screen shots he shared of this:

Google Sitelinks Icons

When we tried to do this we haven’t seen same result so we think Google is testing this.

How to optimise url structure for best seo practice

Web page accessDoes the structure of a URL (or uniform resource locator for pointless trivia fans) matter to SEO? Yes it does, in fact there are many best practices you should consider when creating a URL for your content.

1) Keep a simple, readable structure

This is Google’s number one most important advice – “a site’s URL structure should be as simple as possible.”

It should be logical and readable for human beings. So you’re URL should be www.example.com/SEO-advice-for-beginners not www.example.com/index.php?id_sezione=360&sid=4j3898enno0223bns983201djis03

Use actual words and sentences that anyone can understand, especially when copied into other documents or emails. Stay away from eternally long random patterns of letters and numbers. Nobody wants to click on that.

Gov.UK recommends it should be as short, memorable and unambiguous as possible, especially if a URL is going to be referred to offline.

2) Use hyphens to break up words in a URL

Punctuation is key in promoting readability in URLs. Google recommends hyphens (www.example.com/SEO-advice-for-beginners) instead of underscores (_).

3) All URLs must be in lower case

If your URL contains upper case letters, redirect to the lower case version. In some cases (if you’re hosting with Linux/Unix servers) identical URLs where the sole difference is a capital letter – example.com/webpage versus example.com/webPage – can be considered different pages.

4) Stop-words in URLs

It used to be that you were recommended to avoid stop words (a, an, the) in URLs, but that doesn’t matter anymore. A URL just needs to make sense to human eyes.

5) Your headline doesn’t have to match the URL exactly

In fact it’s a good idea to vary the text, and make it more concise. If your headline says ’25 super-useful SEO best practice tips for beginners’ it may be useful to pair it with a simpler URL: 25-SEO-best-practice-tips-for-beginners

6) Make sure your keywords are near the front of a URL

It’s still good SEO practice to ensure a page’s keywords are near the front of a URL – but it still needs to be readable AND not stuffed with keywords

7) Use a single domain or subdomain

According to Moz, “a company blog is far more likely to perform well in the rankings and to help the rest of your site’s content perform well if it’s all together on one sub and root domain.”

There’s apparently plenty of evidence to suggest that when a company moves content from a subdomain to a subfolder, they see a positive boost in search visibility in traffic.

8) The fewer folders (slashes) the better

Again, according to Moz, the more slashes your URL has, won’t necessarily harm your performance, but it can create an illusion of depth and make indexing your content more complex.

9) URLS should be the verb stem

As recommended by Gov.uk – you should use the term ‘apply’ rather than ‘applying’ for instance.

10) Avoid high numbers of URLs that point to identical or similar content

Overly complex URLs with multiple parameters (such as in point number one) can cause problems for Googlebots, by creating too many different URLs containing similar content.

Google provides a huge list of how this problem can be created in its guide as mentioned in my introduction. It includes:

  • Additive filtering of a set of items. If you provide different views of the same set of items or search results, especially if you let users filter by a certain criteria in an additive manner (for example: hotels in New York and with a panoramic view), the number of URLs on your site will “explode.”
  • Dynamic generation of documents
  • Problematic parameters in the URL (such as session IDs)
  • Sorting parameters
  • Irrelevant parameters in the URL, such as referral parameters
  • Dynamically generated calendar

How to fix URL problems

Here are Google’s recommendations for fixing problematic URLs:

  • Use a robots.txt file to block Googlebot’s access to certain URLs. Such as dynamic URLs or URLs that generate search results.
  • Avoid the use of session IDs in URLs. Use cookies instead.
  • Shorten URLs by trimming unnecessary parameters.
  • If your site has an infinite calendar, add a nofollow attribute to links to dynamically created future calendar pages.
  • Check your site for broken relative link

Complete Twitter Card Guide from Beginning to End

https://bufferblog-wpengine.netdna-ssl.com/wp-content/uploads/2014/08/mgyseK4.jpg

We discussed open graph in previous post this post is on twitter card.

We’re all pretty familiar with Twitter’s 140 characters—and of course, a photo is always an eye-catching addition. But what if your Twitter audience could sign up for your email list without ever leaving Twitter, or directly download your new app straight from a tweet? What if a photo and article summary could travel alongside every post of your content?

And what if you could do all of this for free, right now?

Twitter cards offer all this potential and more. In this post, we’ll go over everything a marketer or small business owner needs to know about Twitter cards—how to understand which types are best for you, how to set up and verify your cards, how to analyze and measure your cards and a few caveats and best practices to know to make sure your Twitter cards stand out from the crowd.

Let’s get going!

Exclusive Bonus: Download a free Twitter Cards cheat sheet!Twitter cards

What are Twitter cards?

If you’ve ever watched a Vine, viewed a YouTube video or clicked on a photo with your Twitter stream, you’ve already come into contact with a Twitter card.

Twitter’s card infrastructure is what allows us to have these rich media experiences that go far beyond a 140-character written message right within Twitter.

By adding a few lines of HTML to your webpage (don’t worry, we’ll cover that; it’s probably easier than you think), you can create this experience for your audience, too.

Any users who Tweet links to your content will have a “card” added to the tweet that’s visible to all of their followers. For instance, here’s how The Verge’s Twitter card carried its information right into my Twitter stream when I shared a recent article there. summary card traveling with link

With 9 different card types to choose from, you can use Twitter cards to ask your audience to do things like:

  • Sign up for an email list
  • Register at a website
  • View and use a coupon
  • Visit a landing page
  • Download an app

…without them ever having to leave Twitter. You’ll also get added benefits, like:

  • A consistent look for your posts across platforms (anyone who shares content that has Twitter cards code in place will produce the same media-rich tweets)
  • Consistent attribution that could drive more more traffic to your site and increase the number of people following you
  • Custom titles and descriptions for your photo or URL (a lot of extra room above and beyond the tweet’s standard 140!), and
  • A really awesome mobile experience for your audience.

Are you in yet? Good; now let’s figure out what Twitter cards are right for you.

Types of Twitter cards and their uses

Right now there are 10 types of Twitter cards that cover a lot of different ground for different types of Twitter publishers as well as different marketing goals. Here’s a quick rundown of each type with an example of how that card looks in the Twitter stream.

Summary card

Buffer summary card

Summary cards are Twitter’s “default” card and include a title, description, thumbnail image, Twitter account attribution and a direct link to the content. These are great for blog posts!

Large photo summary card

large photo summary card 2

Large photo summary cards have all the same features of a regular summary card, but they trade a bit of the description space for a big, beautiful image. Highly visual content producers might want to think about this card.

Product card

For anyone working in e-commerce, product cards are going to be very attractive. In addition to a title, description of up to 200 words, thumbnail image and Twitter handle attribution for your product, this card has a spot for selling, with details like price and availability. And remember, once cards are hooked up, this information travels with every tweet. product card

Player card

If you work in music, video, or multimedia, a player card is a must. This card allows your audience to watch, listen or click through your media without leaving Twitter. Think YouTube, SoundCloud, Vine, etc. Here’s how Storify’s card allows users to scroll through all the posts from a recent #Bufferchat while staying put within Twitter. player card Storify

Audio card

Similar to the player card, Twitter’s audio card provides way for you to experience audio directly on Twitter.  With the audio card, you can listen to audio files directly in your timeline on both iOS and Android devices with a single tap. You can also “dock” the audio card and keep listening even as you keep scrolling inside the Twitter app.

twitter audio card

 

Photo card

Pretty similar to tweeting a post with an attached image, a photo card puts the image front and center—great for art, photography or other visual industries. There’s also a field for a photo caption and a link back to your site. Flickr photo card

Gallery card

To display more than one image in a single tweet, Twitter’s gallery card is the answer—this card shows a collection of 4 photos within a tweet. Clicking on any one of these photos expands it to full size. Screen Shot 2014-11-20 at 4.26.26 PM Use this card to tell a story, explain a process or show various views of a scene of product.

App card

For driving traffic directly to an app, the Twitter app card is ideal—particularly on mobile. When an app with cards enabled is mentioned on Twitter, this card shows the app’s name, ratings, price and icon, along with a short description. The link takes viewers directly to the App Store to download. The call-to-action only displays in the Twitter for iPhone, iPad, and Android apps, and only when the user does not already have your app installed. app card mobile

Lead generation card

Looking for leads? Want to grow your email list? Twitter’s lead generation card can help. This card allows users to sign up for a special offer or opt in to a newsletter right from Twitter—Twitter even automagically fills in your email address so all you have to do is say yes. These cards stand alone; they can’t be attached to specific content or URLs right now. Mailchimp lead gen card

You don’t need to add code to your website to create a lead generation card; just go to the Twitter Ads Dashboard and use them anywhere you make a tweet on Twitter. Pro tip for Buffer users: Once you’ve created your lead generation card, you plug the URL Twitter provides you into your Buffer queue!

Website card

Want to motivate your audience to check out a landing page or simply visit your website? A website card could be the answer. These have a unique design that doesn’t get truncated in users’ streams (more on that in a minute) so they’re sure to stand out. Website cards include a horizontal image, space for a bit of copy, a link to your site, and a call-to-action button. Adidas website card

Twitter describes website cards, the social network’s newest offering, as:

A new way for advertisers to easily surface website content within a Tweet and drive relevant traffic to any page of their site, such as their home page, product page, or an important blog post.

You don’t need to add code to your website to create a website card; just go to the Twitter Ads Dashboard and use them anywhere you make a tweet on Twitter.

How to set up Twitter cards

Have you picked your favorite Twitter card options yet? Excellent; now it’s time to set them up. There are a few different ways to go about this, so we’ll do it Choose Your Own Adventure style. (Note: For website and lead generation cards, it’s simple: Just go to the Twitter Ads Dashboard to make them).

If you’re a developer (or are buddies with a developer)

For maximum control and flexibility over your Twitter cards, you can add the appropriate meta tags for the card of your choice onto your site. Twitter offers some nice documentation on how to do this, with the code all set for you to plug your information into. Here’s a sample of their code for a summary card. Twitter sample summary code

If you’re not a developer

Don’t worry if you’re not a developer or can’t wrangle one to give you a hand; there’s still quite a bit you can do on your own. Next step!

If you have a WordPress.org site

There are three different plugins you can try Twitter recommends three plugins to integrate your WordPress site with Twitter Cards: Jetpack, JM Twitter Cards and WordPress SEO by Yoast. Jetpack: Install Jetpack and navigate to the “Social” section to validate your Twitter handle and turn on Twitter cards. Jetpack automatically scans the contents of your post and determines the best card type.

  • If your post has no images, the Twitter card will be set to summary
  • If your post has between 1 and 4 images, the Twitter card will be set to large photo summary
  • If your post has more than 4 images, the Twitter card will be set to gallery

WordPress SEO: Install WordPress SEO by Yoast and navigate to Social > Twitter, where you can choose between summary and large photo summary cards. Bonus: Yoast also helps out with Facebook Open Graph!

JM Twitter Cards: Install JM Twitter Cards and you can enable summary, large photo summary, app, and photo cards. No matter which of these plugins you use, you’ll need to wind up your process by taking a URL from your blog or website (not the main site name; a specific URL where you’d like your card to display) and running it through the Twitter Card validator. You’ll probably end up getting a message like this. card validator screen From here, click the “request approval” button, fill out the quick information the form asks for (your website, a description of your website, your Twitter handle, your email address) request approval

Twitter says it could take up to a few weeks to get approval, but all my tests for this post got approved almost instantly. When you’re approved, you’ll get an email that looks like this: card activation message   The next time you tweet one of your links, it should be card-ified!

If you have a WordPress.com site

By default, WordPress-hosted blogs are enabled for Twitter Cards. WordPress automatically scans the contents of your post and determines the best Card type. (You’ll still need to do the validator step, though.)

If you have a Tumblr or Blogger blog

For Tumblr, Blogger or any other kind of site, get working on befriending a developer or getting comfortable with code—adding meta tags are the way to go here (you’ll also need to validate). This also goes for more complicated cards that require a few extra steps, like the player card.

Need-to-knows and strategies for Twitter cards

So far we’ve talked about all the great things about Twitter cards, of which there are many. There are a few caveats to know, too.

1. Most cards are unexpanded, so make users want to click

First of all, your Twitter card ensures that all your extra information will travel with every post to Twitter. However, there’s no guarantee that your audience will click to see it. For instance, here’s how the Huffington Post would likely want you see their post. huffington post photo summary card

But in the Twitter stream, and as it gets retweeted by fans, here’s how it looks. Only when that “view summary” text at the bottom right is clicked will users see the full Twitter card. unexpanded photo summary card

The default for all Twitter cards (except website and lead generation cards, which are treated a little differently) is this unexpanded version that relies on the user to click for more. The lesson here? Make your copy so compelling and intriguing that users want to click (not so different than what we’re already used to doing with Twitter, right?) One great example I came across is Etsy. Their tweets are short and sweet and do a great job of compelling the viewer to click. Etsy product cards with curiosity gap copy

What aren’t they getting out of? Let’s find out: Etsy full product card

Ah, got it. Looks comfy! Basically, whichever card you choose you’ll want to be aware that every word counts—in the tweet itself and in your extra space for descriptions and calls-to-action. Make your offer clear, your text compelling. You might even consider including a “click to expand” call to action in your tweet to ensure maximum visibility.

Prioritize images

Twitter is pretty good about finding and resizing images to fit your posts, but help yourself out by devoting a little extra time to your images in posts where cards are enabled and the images you make for special cards like lead generation and website cards. Once your cards are in place it might be tempting to let them to do the work and tweet fewer photos, but keep prioritizing attached photos, too! Tweets with images attract 18% more clicks, 89% more favorites and 150% more retweets than those without them, and Twitter shows photos by default in the stream (as opposed to cards, which need to be clicked to expand). A quick glance at the metrics for these two Flickr posts shows that the photo your audience sees by default has more of an impact than the one they have to click to find. Flickr--attached photo and photo card

Test your cards

With so many Twitter card options, I bet you knew we wouldn’t let you leave this post without talking about A/B testing. Try creating multiple cards for the same campaign with varying images, text and calls-to-action to see which types work best. You might even consider multiple types of cards. Depending on your content and goals, you could find that various sections of your site thrive with different card types.

Cards + pins + ads

With cards plus the ability to pin tweets to the top of your timeline, Twitter has given marketers a powerful one-two punch. We’ve had particular success at Buffer with pinning our lead generation card to the top of our Twitter stream. card plus pin

Marketers with an advertising budget cam combine lead generation or website cards with targeting options like interest, keywords, or tailored audiences segments to drive qualified traffic at just the right time

How to measure success with Twitter cards

I imagine after you’ve gone through all the effort of choosing your cards, bribing a developer and/or installing plugins, and crafting killer cards, you’d like to know how Twitter cards are performing for you. Twitter’s got you covered! Let’s take a look at some of the analytics you’ll find in your account. To see your analytics, navigate from your Twitter account to Ads:

get to Twitter ads

Then to Analytics > Twitter Cards:

get to Twitter card analytics

Once there, you should see a dashboard that looks something like this: Twitter cards analytics dashboard

Since the information in cards travel with the tweet regardless of who’s posting, these metrics are for all Twitter users (not just your tweets). Twitter can’t count posts made through third party tools, though.

Top line metrics

These are the numbers at the very top and include: URL clicks: The number of clicks on a URL in a tweet or a card Install attempts: The number of times users who don’t already have the app installed clicked “Get the app” on a card that has app information enabled Retweets: Retweets that occur on tweets containing a link to your content

Snapshot

The snapshot graphic is a quick look at how tweets and cards are driving traffic, installs or shares—click on the circle of the goal that interests you to see it in full.

Change over time

This graph has the same impressions, tweets and clicks metrics from snapshot, but formed into a rolling 28-day look at how each metric has grown in comparison to the others. change over time

Card types

This chart allows you to benchmark your Twitter cards by comparing their performance against overall standards. card types

Links

See which content is faring best with this chart, which surfaces your top-performing posts by impressions and clicks (up to 100 top links). Twitter cards links analytics

Influencers

Whose posts had the most impact on your impressions and clicks? This chart will tell you–all the way up to your top 100 influencers.Twitter cards influencers

Tweets

Here’s a useful list of the top posts that drove clicks back to your site (note that they’re not necessarily coming from your account alone)

Sources

Where are all the posts that include a URL or card of yours coming from? This chart captures all the different apps, tools and widgets your audience is posting from. Twitter cards sources

Google has introduced new webpage tester in search console

Accelerated Mobile Pages (AMP) is a great way to make content on your website accessible in an extremely fast way. To help ensure that your AMP implementation is working as expected , Search Console now has an enhanced AMP testing tool.

.

This testing tool is mobile-friendly and uses Google’s live web-search infrastructure to analyze the AMP page with the real Googlebot. The tool tests the validity of the AMP markup as well as any structured data on the page. If issues are found, click on them to see details, and to have the line in the source-code highlighted. For valid AMP pages, we may also provide a link to a live preview of how this page may appear in Google’s search results.

With the share button on the bottom right, you can now share a snapshot of the results that you’re currently seeing with others. This makes it easier to discuss issues with your team, whether they’re regular occurrences or one-time quirks that you need to iron out. Just click the share button and pass on the URL for this test snapshot. This share feature is now also available in the mobile-friendly testing tool.

Detailed guide on hreflang

HreflangHreflang is a technical solution for sites that have similar content in multiple languages. A site owner wants search engines to point people to the most “fitting” language. Say a user is Dutch, the page that ranks is English, but there’s also a Dutch version. You would want Google to show the Dutch page in the search results for that user. This is the type of problem hreflang was designed to solve.

What is hreflang meant for?

hreflang is a method to mark up pages that are similar in meaning but aimed at different languages and/or regions. You can use this for three types of variations:

  • Content with regional variations like en-us and en-gb.
  • Content in different languages like en, de and fr.
  • A combination of different languages and regional variations.

You can use hreflang to target different markets that use the same language. This is a fairly common use case. Using hreflang you can differentiate between the US and the UK, or between Germany and Austria.

What’s the SEO benefit of hreflang?

So why are we even talking about hreflang? What is the SEO benefit? There are two main reasons, from an SEO point of view, why you should implement it.

First and foremost, if you have a version of a page that you have optimized for the users language and location, you want them to land on that page. Having the right language and location dependent information improves their user experience and thus leads to fewer people bouncing back to the search results. Fewer people bouncing back to the search results leads to higher rankings.

The second reason is that hreflang prevents a duplicate content problem. You might have the same content in English on different URLs aimed at the UK, the US and Australia. The difference on these pages might be as small as a change in prices and currency. Google might not understand on its own what you’re trying to do and see it as duplicate content. With hreflang you make it very clear to the search engine that it’s (almost) the same content, just optimized for different people.

What is hreflang?

hreflang is code, which you can show to search engines in three different ways, more on that below. With this code you specify all the different URLs on your site(s) that have the same content. These URLs can have the same content in a different language, or the same language but targeted at a different region.

What does hreflang accomplish?

Who supports hreflang?

hreflang is supported by Google and Yandex. Bing doesn’t have an equivalent, but does support language meta tags.

In a complete hreflang implementation, every URL specifies which other variations are available. When a user searches, Google goes through the following process:

  1. it determines that it wants to rank a URL;
  2. it checks whether that URL has hreflang annotations;
  3. it presents the searcher with the results with the most appropriate URL for that user.

The users current location and his language settings determine the most appropriate URL. A user can have multiple languages in his browser’s settings. I, for instance, have Dutch, English and German in there. The order in which these languages appear in my settings determines the most appropriate language.

Should you use hreflang?

Tip: homepage first!

If you’re not sure on whether you want to implement hreflang on your entire site, start with your homepage! People searching for your brand will get the right page. This is a lot easier to implement and it will “catch” a large part of your traffic.

Based on what we’ve learned on what hreflang is and how it works, we can determine if you should use it. You should use it if:

  • you have the same content in multiple languages;
  • you have content aimed at different geographic regions, but in the same language.

Whether the content you have resides on one domain or multiple domains does not matter. You can link variations within the same domain but can also link between domains.hreflang: the ultimate guide

Architectural implementation choices

One thing is very important when implementing hreflang: don’t be too specific! Let’s say you have three types of pages:

  • German
  • German, specifically aimed at Austria
  • German, specifically aimed at Switzerland

You could choose to implement them using three hreflang attributes like this:

  • de-de targeting German speakers in Germany
  • de-at targeting German speakers in Austria
  • de-ch targeting German speakers in Switzerland

However, which of these three results should Google show to someone searching in German in Belgium? The first page would probably be the best. To make sure that every German searching user who does not match either de-at or de-ch gets that one, change that hreflang attribute to just de. Specifying just the language is in many cases a smart thing to do.

It’s good to know that when you create sets of links like this, the most specific one wins. The order in which the search engines sees the links doesn’t matter, it’ll always try to match from most specific to least specific.

Technical implementation basics

Regardless of which type of implementation you choose (more on that below), there are three basic rules.

1. Valid hreflang attributes

The hreflang attribute needs to contain a value that consists of the language, optionally combined with a region. The language attribute needs to be in ISO 639-1 format (a two letter code).

Wrong region codes

Google can deal with some of the common mistakes with region codes, although you shouldn’t take any chances. For instance, it can deal with en-uk just as well as with the “right” en-gb. However, en-eu does not work, as eu doesn’t define a country.

The region is optional and should be in ISO 3166-1 Alpha 2 format, more specifically, it should be an officially assigned element. This means you need to use this list for verification. This is where things often go wrong: using the wrong region code is a very common problem. Use the linked lists on Wikipedia to verify that you’re using the right region and language codes.

2. Return links

The second basic rule is about return links. Regardless of your type of implementation, each URL needs return links to every other URL, note that it should point at the canonical versions, more on that below. The more languages you have the more you might be tempted to limit those return links: don’t. If you have 80 languages, you’ll have hreflang links for 80 URLs. There’s no getting around that.

3. hreflang link to self

The third and final basic rule is about self-links. Just like those return links might feel weird at some point, the hreflang link to the current page feels weird for some developers. It’s required though and not having it will mean your implementation will not work.

Technical implementation choices

There are three ways to implement hreflang: using link elements in the <head>, using HTTP headers or using an XML sitemap. Each has its uses, so we’ll discuss them and explain why you should choose any of these.

1. HTML hreflang link elements in your <head>

The first method to implement hreflang we’ll discuss is HTML hreflang link elements. To implement hreflang using header link elements, you add code like this to the <head> section of every page:

<link rel="alternate" href="http://example.com/" 
      hreflang="en" />
<link rel="alternate" href="http://example.com/en-gb/" 
      hreflang="en-gb" />
<link rel="alternate" href="http://example.com/en-au/" 
      hreflang="en-au" />

As every variation needs to link to every other variation, these implementations can become quite big and lead to performance issues. If you have 20 languages, choosing HTML link elements would mean adding 20 link elements as shown above to every page. This means adding 1.5KB on every page load, that no user will ever use, but will have to download. On top of that, your CMS will have to do several database calls to generate all these links. This markup is purely meant for search engines. That’s why I would not recommend this for larger sites, as it adds far too much, unneeded, overhead.

2. hreflang HTTP headers

The second method of implementing hreflang is through HTTP headers. HTTP headers are the solution for all your PDFs and other non-HTML content you might want to optimize. Link elements work nicely for HTML documents, but not for other types of content as you can’t include them. That’s where HTTP headers come in. They should look like this:

Link: <http://es.example.com/document.pdf>; 
rel="alternate"; hreflang="es", 
<http://en.example.com/document.pdf>; 
rel="alternate"; hreflang="en", 
<http://de.example.com/document.pdf>; 
rel="alternate"; hreflang="de"

The problem with having a lot of HTTP headers is similar to the problem with link elements in your <head>: it adds a lot of overhead to every request.

3. An XML sitemap hreflang implementation

The third option to implement hreflang is using XML sitemap markup. It uses the xhtml:link attribute in XML sitemaps to add the annotation to every URL. It works very much in the same way as you would in a page’s <head> with <link> elements. If you thought link elements were verbose, the XML sitemap implementation is even worse. This is the markup needed for just one URL with two other languages:

<url>
  <loc>http://www.example.com/uk/</loc> 
  <xhtml:link rel="alternate" hreflang="en" 
     href="http://www.example.com/" /> 
  <xhtml:link rel="alternate" hreflang="en-au" 
     href="http://www.example.com/au/" /> 
  <xhtml:link rel="alternate" hreflang="en-gb" 
     href="http://www.example.com/uk/" />
</url>

You can see it has a self-referencing URL as the third URL, specifying the specific URL is meant for en-gb, and it specifies two other languages. Now, both other URLs would need to be in the sitemap too, looking like this:

<url>
  <loc>http://www.example.com/</loc> 
  <xhtml:link rel="alternate" hreflang="en" 
     href="http://www.example.com/" /> 
  <xhtml:link rel="alternate" hreflang="en-au" 
     href="http://www.example.com/au/" /> 
  <xhtml:link rel="alternate" hreflang="en-gb" 
     href="http://www.example.com/uk/" />
</url>
<url>
  <loc>http://www.example.com/au/</loc> 
  <xhtml:link rel="alternate" hreflang="en" 
     href="http://www.example.com/" /> 
  <xhtml:link rel="alternate" hreflang="en-au" 
     href="http://www.example.com/au/" /> 
  <xhtml:link rel="alternate" hreflang="en-gb" 
     href="http://www.example.com/uk/" />
</url>

As you can see, basically we’re only changing the URLs within the <loc> element, as everything else needs to be the same. This way, each URL has a self-referencing hreflang attribute, and return links to the appropriate other URLs.

XML sitemap markup like this is very verbose: a lot of output is needed to do this for a lot of URLs. The benefit of an XML sitemap implementation is simple: your normal users won’t be bothered with this markup. This has the benefit of not adding extra page weight and it doesn’t require a lot of database calls on page load to generate this markup.

Another benefit of adding hreflang through the XML sitemap is that it’s usually a lot easier to change an XML sitemap than to change all the pages on a site. No need to go through large approval processes and maybe you can even get direct access to the XML sitemap file.

Other technical aspects of an hreflang implementation

We’re going to assume that you’ve decided which type of technical implementation you’re going to choose. There are a couple of other technical specificities you should know about before you start implementing hreflang.

hreflang x-default

There is a special hreflang attribute value that’s called x-default. The x-default value specifies where a user should be sent if none of the languages you’ve specified in your other hreflang links match their browser settings. In a link element it looks like this:

<link rel="alternate" href="http://example.com/" 
  hreflang="x-default" />

When it was introduced, it was explained as being for “international landing pages”, ie pages where you redirect users based on their location. However, it can basically be described as the final “catch-all” of all the hreflang statements. If the users location and language didn’t match anything else, that’s where they will be sent.

In the German example we mentioned above, a user searching in English still wouldn’t have a “fitting” URL. That’s one of the cases where x-default comes into play. You’d add a fourth link to the markup, and end up with these 4:

  • de
  • de-at
  • de-ch
  • x-default

In this case, the x-default link would point to the same URL as the de one. We would not encourage you to remove the de link though, even though technically that would create exactly the same result. In the long run it’s usually better to have both as it specifies what language that de page is in and makes the code easier to read.

hreflang and rel=canonical

rel=”canonical”

If you don’t know what rel="canonical" is, read this article!

rel="alternate" hreflang="x"markup and rel="canonical" can and should be used together. Every language should have a rel="canonical" link pointing to itself. In the first example, this would look like this, assuming that we’re on the example.com homepage:

<link rel="canonical" href="http://example.com/">
<link rel="alternate" href="http://example.com/" 
      hreflang="en" />
<link rel="alternate" href="http://example.com/en-gb/" 
      hreflang="en-gb" />
<link rel="alternate" href="http://example.com/en-au/" 
      hreflang="en-au" />

If we were on the en-gb page, not all that much would change other than the canonical:

<link rel="canonical" href="http://example.com/en-gb/">
<link rel="alternate" href="http://example.com/" 
      hreflang="en" />
<link rel="alternate" href="http://example.com/en-gb/" 
      hreflang="en-gb" />
<link rel="alternate" href="http://example.com/en-au/" 
      hreflang="en-au" />

Don’t make the mistake of setting the canonical on the en-gb page to http://example.com/, as this breaks the implementation. It’s very important that the hreflang links point to the canonical version of each URL. These systems should work hand in hand!

Useful tools when implementing hreflang

If you’ve come this far, you’ll probably be thinking “wow this is hard”, I know I thought that while learning about the topic. Luckily, there are quite a few tools available for people who dare to start implementing hreflang.

hreflang tag generator

hreflang tag generator tool by Aleyda

Aleyda Solis, who has written quite a lot about this topic too, has created a very useful hreflang tag generator that helps you generate link elements. Even when you’re not choosing for a link element implementation, this can be useful to create some example code.

hreflang XML sitemap generator

The Media Flow have created an hreflang hreflang XML sitemap generator. You can feed it a CSV with URLs per language and it creates an XML sitemap. A very good first step when you decide to go this route.

The CSV file you feed this XML sitemap generator needs columns for every language. If you want to add an x-default URL to it as well, just create a column called x-default.

Keeping hreflang alive: process

Once you’ve created a working hreflang setup, you need to set up processes. It’s probably also a good idea to regularly audit your implementation to make sure it’s still set up correctly.

Make sure that people in your company who deal with content on your site know about hreflang. This makes sure they won’t do things that break your implementation. Two things are very important:

  1. When a page is deleted, check whether its counterparts are updated.
  2. When a page is redirected, change the hreflang URLs on its counterparts.

If you do that and audit regularly, you shouldn’t run into many issues.

Conclusion

Setting up hreflang is a cumbersome process. It’s a tough standard with a lot of specific things you should know and deal with. This guide will be updated as new things are introduced around this specification and best practices evolve, so check back when you’re working on your implementation again!

When you will get HTTPS ranking boost

This is not new news, we knew HTTPS was the only signal back in 2014 and validity doesn’t matter.

Google will give the ranking boost to HTTPS pages even if they are not secure. Be it mixed content errors, broken certificates, etc.

Gary Illyes of Google said on the HTTPS signal, “basically looking at the first five characters in front of the URL, and if it’s HTTPS and it managed to get in the search results and it will get a minimal boost.”

And if the HTTPS is not listed, it won’t get the boost: