About a month ago, I was informed by Baidu that they would be releasing a game-changer to rebuild the eco-system of the web in China. On March 24, in alignment with their previously published Web Ads Guideline (guidelines are in Chinese) Baidu Webmaster Tools announced that they have added a new crawling spider.
This new spider will not only crawl the HTML of the web pages but also render the page with other elements including CSS, JS, and images to help Baidu better understand the content of the page and provide more meaningful results. Baidu started the test on the 23rd and they are rolling it out now. The user agent of spider is marked as:
Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html) (Desktop)
Mozilla/5.0 (iPhone; CPU iPhone OS 9_1 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 Mobile/13B143 Safari/601.1 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html) (Mobile)
Baidu said this change is adding some pressure for sites, as it costs extra resources, but is advising webmasters not to block those spiders.
This change makes Baidu more like other advanced search engines and signals an elevated investment on the user experience and safety of the web. During the rollout, we expect the results rankings on Baidu to experience increased fluctuation. For technical SEO, Merkle is updating its Fetch & Render Tool to reflect this major change.
In 2015, Merkle became Baidu’s first US-headquartered reseller. For more on our partnership with Baidu and how this can benefit your China program, visit our Baidu page.
