首页 > 代码库 > MapReduce单元测试
MapReduce单元测试
MapReduce进行单元测试的步骤:
1. 在POM中添加MRUnit
<dependency> <groupId>org.apache.mrunit</groupId> <artifactId>mrunit</artifactId> <version>1.1.0</version> <classifier>hadoop2</classifier> <scope>test</scope> </dependency>
2. Mapper测试用例
@Test public void testCountMapper() throws IOException { LongWritable key = new LongWritable(0); Text value = new Text("hadoop yarn"); new MapDriver<LongWritable,Text,Text,IntWritable>() .withMapper(new WordCountMapper()) .withInput(key,value) .withOutput(new Text("hadoop"),new IntWritable(1)) .withOutput(new Text("yarn"),new IntWritable(1)) .runTest(); }
3. Reducer测试用例
@Test public void testCountReducer() throws IOException { new ReduceDriver<Text,IntWritable,Text,IntWritable>() .withReducer(new WordCountReducer()) .withInput(new Text("hadoop"), Arrays.asList(new IntWritable(1),new IntWritable(1))) .withOutput(new Text("hadoop"),new IntWritable(2)) .runTest(); }
MapReduce单元测试
声明:以上内容来自用户投稿及互联网公开渠道收集整理发布,本网站不拥有所有权,未作人工编辑处理,也不承担相关法律责任,若内容有误或涉及侵权可进行投诉: 投诉/举报 工作人员会在5个工作日内联系你,一经查实,本站将立刻删除涉嫌侵权内容。