首页 > 代码库 > MRUnit的安装和使用
MRUnit的安装和使用
MRUnit是对MapReduce程序进行单元测试的工具,可以对Mapper和Reducer程序分别进行测试。但是它没有集成在Hadoop安装环境中,如果想在开发MapReduce程序时使用这个工具,就需要自己安装。
MRUnit的安装
安装环境:
Eclipse版本为3.6.0
Hadoop版本为1.0.4
安装步骤:
(1)下载MRUnit,网址为http://mrunit.apache.org/,我下载的是 apache-mrunit-1.0.0-hadoop1-bin.tar.gz
(2)解压缩下载的文件,将lib目录下的hamcrest-core-1.1.jar,junit-4.10.jar,mockito-all-1.8.5.jar和mrunit-1.0.0-hadoop1.jar加入到Eclipse的项目中。
方法如下:选中待测试项目-->右键Buid Path-->Configure Buid Path-->点击Libraries-->点击Add External JARs
MRUnit的使用
以Hadoop权威指南上的MaxTemperature程序为例,整个项目中包括如下4个源文件,前两个分别是Mapper程序和Reducer程序,后两个分别是针对Mapper和Reducer的测试程序:MaxTemperatureMapper.java,MaxTemperatureReducer.java,MaxTemperatureMapperTest.java,MaxTemperatureReducerTest.java
MaxTemperatureMapper.java
import java.io.IOException; import org.apache.hadoop.io.IntWritable; import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Mapper; public class MaxTemperatureMapper extends Mapper<LongWritable, Text, Text, IntWritable> { private static final int MISSING = 9999; public void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException { String line = value.toString(); String year = line.substring(15, 19); int airTemperature; if(line.charAt(87) == '+') airTemperature = Integer.parseInt(line.substring(88, 92)); else airTemperature = Integer.parseInt(line.substring(87, 92)); String quality = line.substring(92, 93); if(airTemperature != MISSING && quality.matches("[01459]")) context.write(new Text(year), new IntWritable(airTemperature)); } }
import java.io.IOException; import org.apache.hadoop.io.IntWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Reducer; public class MaxTemperatureReducer extends Reducer<Text, IntWritable, Text, IntWritable> { public void reduce(Text key, Iterable<IntWritable> values, Context context) throws IOException, InterruptedException { int maxValue = http://www.mamicode.com/Integer.MIN_VALUE;>MaxTemperatureMapperTest.java
import java.io.IOException; import org.apache.hadoop.io.Text; import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.IntWritable; import org.apache.hadoop.mrunit.mapreduce.MapDriver; import org.junit.Test; public class MaxTemperatureMapperTest { @Test public void processesValidRecord() throws IOException, InterruptedException { Text value = http://www.mamicode.com/new Text("0057332130999991950010103004+51317+028783FM-12+017199999V0203201N00721004501CN0100001N9-01281-01391102681");>MaxTemperatureReducerTest.java
import java.io.IOException; import java.util.Arrays; import org.junit.Test; import org.apache.hadoop.io.Text; import org.apache.hadoop.io.IntWritable; import org.apache.hadoop.mrunit.mapreduce.ReduceDriver; public class MaxTemperatureReducerTest { @Test public void returnsMaximumIntegerInValues() throws IOException, InterruptedException { new ReduceDriver<Text, IntWritable, Text, IntWritable>() .withReducer(new MaxTemperatureReducer()) .withInput(new Text("1950"), Arrays.asList(new IntWritable(10), new IntWritable(5))) .withOutput(new Text("1950"), new IntWritable(10)) .runTest(); } }
需要注意程序中的@Test是不可或缺的
下面就可以运行测试程序了,如果要测试Mapper程序,那么就选中MaxTemperatureMapperTest.java,右键Run As-->JUnit Test,如果进度条为绿色,则表示测试正确,否则代表有错误。测试Reducer程序也是如此。
MRUnit的安装和使用