首页 > 代码库 > spark1.统计句子中特定内容
spark1.统计句子中特定内容
val logFile = "./README.md" // Should be some file on your server.
val conf = new SparkConf().setAppName("Simple Application").setMaster("local")
val sc = new SparkContext(conf)
val logData = http://www.mamicode.com/sc.textFile(logFile, 2).cache()
// val numAs = logData.filter(line => line.contains("h")).count()
// val numBs = logData.filter(line => line.contains("j")).count()
var params = List("h","j","c","w");
var searchAnylisay = params.map(item => logData.filter(line => line.contains(item)).count() )
println("searchAnylisay length : %s,".format(searchAnylisay.length))
searchAnylisay.foreach( x => println(x))
spark1.统计句子中特定内容
声明:以上内容来自用户投稿及互联网公开渠道收集整理发布,本网站不拥有所有权,未作人工编辑处理,也不承担相关法律责任,若内容有误或涉及侵权可进行投诉: 投诉/举报 工作人员会在5个工作日内联系你,一经查实,本站将立刻删除涉嫌侵权内容。