首页 > 代码库 > Spark2 DataSet 创建新行之flatMap
Spark2 DataSet 创建新行之flatMap
val dfList = List(("Hadoop", "Java,SQL,Hive,HBase,MySQL"), ("Spark", "Scala,SQL,DataSet,MLlib,GraphX")) dfList: List[(String, String)] = List((Hadoop,Java,SQL,Hive,HBase,MySQL), (Spark,Scala,SQL,DataSet,MLlib,GraphX)) case class Book(title: String, words: String) val df=dfList.map{p=>Book(p._1,p._2)}.toDS() df: org.apache.spark.sql.Dataset[Book] = [title: string, words: string] df.show +------+--------------------+ | title| words| +------+--------------------+ |Hadoop|Java,SQL,Hive,HBa...| | Spark|Scala,SQL,DataSet...| +------+--------------------+ df.flatMap(_.words.split(",")).show +-------+ | value| +-------+ | Java| | SQL| | Hive| | HBase| | MySQL| | Scala| | SQL| |DataSet| | MLlib| | GraphX| +-------+
Spark2 DataSet 创建新行之flatMap
声明:以上内容来自用户投稿及互联网公开渠道收集整理发布,本网站不拥有所有权,未作人工编辑处理,也不承担相关法律责任,若内容有误或涉及侵权可进行投诉: 投诉/举报 工作人员会在5个工作日内联系你,一经查实,本站将立刻删除涉嫌侵权内容。