首页 > 代码库 > HtmlUnit模仿浏览器抓取数据(含ajax)
HtmlUnit模仿浏览器抓取数据(含ajax)
import java.io.IOException; import java.net.MalformedURLException; import com.gargoylesoftware.htmlunit.BrowserVersion; import com.gargoylesoftware.htmlunit.FailingHttpStatusCodeException; import com.gargoylesoftware.htmlunit.NicelyResynchronizingAjaxController; import com.gargoylesoftware.htmlunit.SilentCssErrorHandler; import com.gargoylesoftware.htmlunit.WebClient; import com.gargoylesoftware.htmlunit.html.HtmlPage; public class WorldBankCrawl { public static void main(String[] args) throws FailingHttpStatusCodeException, MalformedURLException, IOException { WebClient webClient = new WebClient(BrowserVersion.FIREFOX_24); webClient.setCssErrorHandler(new SilentCssErrorHandler()); webClient.setAjaxController(new NicelyResynchronizingAjaxController()); webClient.getOptions().setCssEnabled(true); webClient.getOptions().setRedirectEnabled(false); webClient.getOptions().setAppletEnabled(false); webClient.getOptions().setJavaScriptEnabled(true); webClient.getOptions().setPopupBlockerEnabled(true); webClient.getOptions().setTimeout(10000); HtmlPage page = webClient.getPage("http://huaban.com/favorite/home/"); System.out.println(page.asXml()); webClient.closeAllWindows(); } }
HtmlUnit模仿浏览器抓取数据(含ajax)
声明:以上内容来自用户投稿及互联网公开渠道收集整理发布,本网站不拥有所有权,未作人工编辑处理,也不承担相关法律责任,若内容有误或涉及侵权可进行投诉: 投诉/举报 工作人员会在5个工作日内联系你,一经查实,本站将立刻删除涉嫌侵权内容。