We use cookies. You have options. Cookies help us keep the site running smoothly and inform some of our advertising, but if you’d like to make adjustments, you can visit our Cookie Notice page for more information.
We’d like to use cookies on your device. Cookies help us keep the site running smoothly and inform some of our advertising, but how we use them is entirely up to you. Accept our recommended settings or customise them to your wishes.

Baidu Adds New Spider with Rendering Job

About a month ago, I was informed by Baidu that they would be releasing a game-changer to rebuild the eco-system of the web in China. On March 24, in alignment with their previously published Web Ads Guideline (guidelines are in Chinese) Baidu Webmaster Tools announced that they have added a new crawling spider.

This new spider will not only crawl the HTML of the web pages but also render the page with other elements including CSS, JS, and images to help Baidu better understand the content of the page and provide more meaningful results. Baidu started the test on the 23rd and they are rolling it out now. The user agent of spider is marked as:

Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html) (Desktop)
Mozilla/5.0 (iPhone; CPU iPhone OS 9_1 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 Mobile/13B143 Safari/601.1 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html) (Mobile)

Baidu said this change is adding some pressure for sites, as it costs extra resources, but is advising webmasters not to block those spiders.

This change makes Baidu more like other advanced search engines and signals an elevated investment on the user experience and safety of the web. During the rollout, we expect the results rankings on Baidu to experience increased fluctuation. For technical SEO, Merkle is updating its Fetch & Render Tool to reflect this major change.

In 2015, Merkle became Baidu’s first US-headquartered reseller. For more on our partnership with Baidu and how this can benefit your China program, visit our Baidu page.