Developing algorithmic changes to search involves a process of experimentation. Part of that experimentation is having evaluators—people who assess the quality of Google’s search results—give us feedback on our experiments. Ratings from evaluators do not determine individual site rankings, but are used help us understand our experiments. The evaluators base their ratings on guidelines we give them; the guidelines reflect what Google thinks search users want.
In 2013, we published our human rating guidelines to provide transparency on how Google works and to help webmasters understand what Google looks for in web pages. Since that time, a lot has changed: notably, more people have smartphones than ever before and more searches are done on mobile devices today than on computers.
We often make changes to the guidelines as our understanding of what users wants evolves, but we haven’t shared an update publicly since then. However, we recently completed a major revision of our rater guidelines to adapt to this mobile world, recognizing that people use search differently when they carry internet-connected devices with them all the time. You can find that update here (PDF).
This is not the final version of our rater guidelines. The guidelines will continue to evolve as search, and how people use it, changes. We won’t be updating the public document with every change, but we will try to publish big changes to the guidelines periodically.
We expect our phones and other devices to do a lot, and we want Google to continue giving users the answers they're looking for—fast!
Two weeks ago, we were extremely lucky to host the 2015 edition of the Top Contributor Summit (#TCsummit), in San Francisco and on Google’s campus in Mountain View, California.
Google Top Contributors are an exceptional group of passionate Google product enthusiasts who share their expertise across our international help forums to support millions of Google users every year. Google’s Top Contributor Summit is an event organised every two years, to celebrate these amazing users. This year we had the pleasure to welcome 526 Top Contributors, from all around the world.
Under the motto “Learn, Connect, Celebrate”, Top Contributors had the chance to learn more about our products, get insights on the future of Google, connect with Googlers and Top Contributors from various products and, finally, to celebrate their positive impact on our products and users.
Footage of the 2015 Top Contributor Summit
We also had the chance to hold Webmaster-specific sessions, which gave Googlers the unique opportunity to meet 56 of our Webmaster Top Contributors, representing 20 countries and speaking 14 different languages.
Group photo of the Webmaster Top Contributor community and the Google Webmaster Relations team
Throughout the day, we had in-depth sessions about Google Webmaster guidelines, Search Console and Google Search. We discussed the most common issues that users are bringing up in our international webmaster forums, and listened to the Top Contributors’ feedback regarding our Search tools. We also talked about the Top Contributor program itself and additional opportunities for our users to benefit from both Google and the TCs’ support. Product managers, engineers and search quality Googlers attended the sessions to listen and bring the feedback given by Top Contributors and users on the forum back to their teams.
Webmaster Top Contributors during the in-depth sessions about Google Webmaster guidelines, Search Console and Google Search
At Google, we are grateful to have the incredible opportunity to meet and connect with some of the most insightful members of the webmaster community and get their feedback on such important topics. It helps us be sure that Google keeps focusing on what really matters to webmasters, content creators, and users.
To learn more about our Top Contributor Program, or to give us your own feedback, visit our Top Contributor homepage or join our Webmaster help forum.
In many cases, it is OK to show slightly different content on different devices. For example, optimizing the smaller space of a smartphone screen can mean that some content, like images, will have to be modified. Or you might want to store your website’s menu in a navigation drawer (find documentation here) to make mobile browsing easier and more effective. When implemented properly, these user-centric modifications can be understood very well by Google.
The situation is similar when it comes to mobile-only redirect. Redirecting mobile users to improve their mobile experience (like redirecting mobile users from example.com/url1 to m.example.com/url1) is often beneficial to them. But redirecting mobile users sneakily to a different content is bad for user experience and is against Google’s webmaster guidelines.
There are cases where webmasters knowingly decide to put into place redirection rules for their mobile users. This is typically a webmaster guidelines violation, and we do take manual action against it when it harms Google users’ experience (see last section of this article).
But we’ve also observed situations where mobile-only sneaky redirects happen without site owners being aware of it:
It's a violation of the Google Webmaster Guidelines to redirect a user to a page with the intent of displaying content other than what was made available to the search engine crawler (more information on sneaky redirects). To ensure quality search results for our users, the Google Search Quality team can take action on such sites, including removal of URLs from our index. When we take manual action, we send a message to the site owner via Search Console. Therefore, make sure you’ve set up a Search Console account.
Be sure to choose advertisers who are transparent on how they handle user traffic, to avoid unknowingly redirecting your own users. If you are interested in trust-building in the online advertising space, you may check out industry-wide best practices when participating in ad networks. For example, the Trustworthy Accountability Group’s (Interactive Advertising Bureau) Inventory Quality Guidelines are a good place to start. There are many ways to monetize your content with mobile solutions that provide a high quality user experience, be sure to use them.
If you have questions or comments about mobile-only redirects, join us in our Google Webmaster Support forum.
tl;dr: We are no longer recommending the AJAX crawling proposal we made back in 2009.
Since the assumptions for our 2009 proposal are no longer valid, we recommend following the principles of progressive enhancement. For example, you can use the History API pushState() to ensure accessibility for a wider range of browsers (and our systems).
Questions and answers
Q: My site currently follows your recommendation and supports _escaped_fragment_. Would my site stop getting indexed now that you've deprecated your recommendation?
A: No, the site would still be indexed. In general, however, we recommend you implement industry best practices when you're making the next update for your site. Instead of the _escaped_fragment_ URLs, we'll generally crawl, render, and index the #! URLs.
Q: Is moving away from the AJAX crawling proposal to industry best practices considered a site move? Do I need to implement redirects?
A: If your current setup is working fine, you should not have to immediately change anything. If you're building a new site or restructuring an already existing site, simply avoid introducing _escaped_fragment_ urls. .
A: In general, websites shouldn't pre-render pages only for Google -- we expect that you might pre-render pages for performance benefits for users and that you would follow progressive enhancement guidelines. If you pre-render pages, make sure that the content served to Googlebot matches the user's experience, both how it looks and how it interacts. Serving Googlebot different content than a normal user would see is considered cloaking, and would be against our Webmaster Guidelines.
If you have any questions, feel free to post them here, or in the webmaster help forum.
Recently we have started rolling out a series of algorithmic changes that aim to tackle hacked spam in our search results. A huge amount of legitimate sites are hacked by spammers and used to engage in abusive behavior, such as malware download, promotion of traffic to low quality sites, porn, and marketing of counterfeit goods or illegal pharmaceutical drugs, etc.
Website owners that don’t implement standard best practices for security can leave their websites vulnerable to being easily hacked. This can include government sites, universities, small business, company websites, restaurants, hobby organizations, conferences, etc. Spammers and cyber-criminals purposely seek out those sites and inject pages with malicious content in an attempt to gain rank and traffic in search engines.
We are aggressively targeting hacked spam in order to protect users and webmasters.
The algorithmic changes will eventually impact roughly 5% of queries, depending on the language. As we roll out the new algorithms, users might notice that for certain queries, only the most relevant results are shown, reducing the number of results shown:
This is due to the large amount of hacked spam being removed, and should improve in the near future. We are continuing tuning our systems to weed out the bad content while retaining the organic, legitimate results. If you have any questions about these changes, or want to give us feedback on these algorithms, feel free to drop by our Webmaster Help Forums.
Posted by Ning Song, Software Engineer
Around ten years ago when we introduced a policy called “First Click Free,” it was hard to imagine that the always-on, multi-screen, multiple device world we now live in would change content consumption so much and so fast. The spirit of the First Click Free effort was - and still is - to help users get access to high quality news with a minimum of effort, while also ensuring that publishers with a paid subscription model get discovered in Google Search and via Google News.
In 2009, we updated the FCF policy to allow a limit of five articles per day, in order to protect publishers who felt some users were abusing the spirit of this policy. Recently we have heard from publishers about the need to revisit these policies to reflect the mobile, multiple device world. Today we are announcing a change to the FCF limit to allow a limit of three articles a day. This change will be valid on both Google Search and Google News.
Google wants to play its part in connecting users to quality news and in connecting publishers to users. We believe the FCF is important in helping achieve that goal, and we will periodically review and update these policies as needed so they continue to benefit users and publishers alike. We are listening and always welcome feedback.
Questions and answers about First Click Free
Q: Do the rest of the old guidelines still apply?
A: Yes, please check the guidelines for Google News as well as the guidelines for Web Search and the associated blog post for more information.
Q: Can I apply First Click Free to only a section of my site / only for Google News (or only for Web Search)?
A: Sure! Just make sure that both Googlebot and users from the appropriate search results can view the content as required. Keep in mind that showing Googlebot the full content of a page while showing users a registration page would be considered cloaking.
Q: Do I have to sign up to use First Click Free?
A: Please let us know about your decision to use First Click Free if you are using it for Google News. There's no need to inform us of the First Click Free status for Google Web Search.
Q: What is the preferred way to count a user's accesses?
A: Since there are many different site architectures, we believe it's best to leave this up to the publisher to decide.
(Please see our related blog post for more information on First Click Free for Google News.)
Thus far in 2015 we have seen a 180% increase in the number of sites getting hacked and a 300% increase in hacked site reconsideration requests. While we are working hard to help webmasters prevent hacks in the first place through efforts such as blog posts and #NoHacked campaigns, we recognize that our reconsideration process is an important part of making recovering from a hack faster and easier. Here's what we've been focussing on:
1) Improved communication
2) Better tools
3) Continuous feedback loop
1. Improving communications with webmasters of hacked sites
Last year we launched the "Note from your reviewer" feature in our reconsideration process. This feature enables us to give specific examples and advice tailored to each case in response to a reconsideration request. Thus far in 2015 we have sent a customized note to over 70% of webmasters whose hacked reconsideration request was rejected, with specific guidance on where and how to find the remaining hacked content. The results have been encouraging, as we've seen a 29% decrease in the average amount of time from when a site receives a hacked manual action to the time when the webmaster cleans up and the manual action is removed.
Example "note from your reviewer" with detailed guidance and a custom example of hacked text and a hacked page
We have also completed our second #NoHacked campaign, with more detailed help on preventing and recovering from hacks. In the campaign, we focused on ways to improve the security on your site as well as ways to fix your site if it was compromised. You can catch up by reading the first post.
2. Better tools including auto-removal of some hacked manual actions
Last year we launched the "Fetch and Render" feature to the Fetch as Google tool, which allows you to see the website exactly as Googlebot sees it. This functionality is useful in recovering from a hack, since many hackers inject cloaked content that's not visible to the normal user but obvious to search engine crawlers like Googlebot.
This year we also launched the Hacked Sites Troubleshooter in 23 languages which guides webmasters through some basic steps to recover from a hack. Let us know if you have found the troubleshooter useful as we're continuing to expand its features and impact.
Finally, we're beta testing the automated removal of some hacked manual actions. In Search Console if Google sees a "Hacked site" manual action under "Partial matches", and our systems detect that the hacked content is no longer present, in some cases we will automatically remove that manual action. We still recommend that you submit a reconsideration request if you see any manual actions, but don't be surprised if a "Hacked site" manual action disappears and saves you the trouble!
Example of a Hacked site manual action on a Partial match: if our systems detect that the hacked content is no longer present, in some cases we will automatically remove the manual action
3. Soliciting your feedback and taking action
Our improved communication and tools have come directly from feedback we've collected from webmasters of sites that have been hacked. For example, earlier this year we hosted webmasters who have been through the hacked reconsideration process in both Mountain View, USA and Dublin, Ireland for brainstorming sessions. We also randomly sampled webmasters that had been through a hacked reconsideration. We found that while only 15% of webmasters were dissatisfied with the process, the main challenges those webmasters faced were in clearer notification of their site being hacked and clearer guidance on how to resolve the hack. This feedback contributed directly our more detailed blog post on hacked recovery, and to much of the content in our latest #NoHacked campaign.
(for hi-res version) https://goo.gl/photos/TkvkwYt23MpVHBwz6
We will continue to support webmasters of hacked sites through the methods detailed above, in addition to the Webmasters help for hacked sites portal and the security, malware & hacked sites section of our forum. And we'd love to hear your ideas in the comments below on how Google can better support webmasters recovering from a hacked website!
Enter your email address:
Delivered by FeedBurner