首页 > 代码库 > [hadoop] hadoop 运行 wordcount
[hadoop] hadoop 运行 wordcount
讲准备好的文本文件放到hdfs中
执行 hadoop 安装包中的例子
[root@hadoop01 mapreduce]# hadoop jar hadoop-mapreduce-examples-2.8.0.jar wordcount /input/ /output/wordcount 17/05/14 02:01:17 INFO client.RMProxy: Connecting to ResourceManager at hadoop01/172.16.253.128:8032 17/05/14 02:01:19 INFO input.FileInputFormat: Total input files to process : 2 17/05/14 02:01:19 INFO mapreduce.JobSubmitter: number of splits:2 17/05/14 02:01:19 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1494742494825_0002 17/05/14 02:01:20 INFO impl.YarnClientImpl: Submitted application application_1494742494825_0002 17/05/14 02:01:20 INFO mapreduce.Job: The url to track the job: http://hadoop01:8088/proxy/application_1494742494825_0002/ 17/05/14 02:01:20 INFO mapreduce.Job: Running job: job_1494742494825_0002 17/05/14 02:01:35 INFO mapreduce.Job: Job job_1494742494825_0002 running in uber mode : false 17/05/14 02:01:35 INFO mapreduce.Job: map 0% reduce 0% 17/05/14 02:02:48 INFO mapreduce.Job: map 100% reduce 0% 17/05/14 02:03:22 INFO mapreduce.Job: map 100% reduce 100% 17/05/14 02:03:25 INFO mapreduce.Job: Job job_1494742494825_0002 completed successfully 17/05/14 02:03:28 INFO mapreduce.Job: Counters: 49 File System Counters FILE: Number of bytes read=350 FILE: Number of bytes written=408885 FILE: Number of read operations=0 FILE: Number of large read operations=0 FILE: Number of write operations=0 HDFS: Number of bytes read=584 HDFS: Number of bytes written=145 HDFS: Number of read operations=9 HDFS: Number of large read operations=0 HDFS: Number of write operations=2 Job Counters Launched map tasks=2 Launched reduce tasks=1 Data-local map tasks=2 Total time spent by all maps in occupied slots (ms)=145615 Total time spent by all reduces in occupied slots (ms)=17369 Total time spent by all map tasks (ms)=145615 Total time spent by all reduce tasks (ms)=17369 Total vcore-milliseconds taken by all map tasks=145615 Total vcore-milliseconds taken by all reduce tasks=17369 Total megabyte-milliseconds taken by all map tasks=149109760 Total megabyte-milliseconds taken by all reduce tasks=17785856 Map-Reduce Framework Map input records=14 Map output records=70 Map output bytes=666 Map output materialized bytes=356 Input split bytes=196 Combine input records=70 Combine output records=30 Reduce input groups=19 Reduce shuffle bytes=356 Reduce input records=30 Reduce output records=19 Spilled Records=60 Shuffled Maps =2 Failed Shuffles=0 Merged Map outputs=2 GC time elapsed (ms)=9667 CPU time spent (ms)=3210 Physical memory (bytes) snapshot=330969088 Virtual memory (bytes) snapshot=6192197632 Total committed heap usage (bytes)=259284992 Shuffle Errors BAD_ID=0 CONNECTION=0 IO_ERROR=0 WRONG_LENGTH=0 WRONG_MAP=0 WRONG_REDUCE=0 File Input Format Counters Bytes Read=388 File Output Format Counters Bytes Written=145
查看执行结果:
[hadoop] hadoop 运行 wordcount
声明:以上内容来自用户投稿及互联网公开渠道收集整理发布,本网站不拥有所有权,未作人工编辑处理,也不承担相关法律责任,若内容有误或涉及侵权可进行投诉: 投诉/举报 工作人员会在5个工作日内联系你,一经查实,本站将立刻删除涉嫌侵权内容。