It has been a day that I am now playing with Hadoop and bumped into issues with Eclipse plugin for Hadoop (Later, I ran mapreduce program on command line, you can find the notes posted here).
I have Windows 7, 64 bit machine, eclipse Indigo 3.7.1, and installed hadoop 0.20.2 (why did I install 0.20.2).
Ok, and then I copied over the eclipse plugin from Hadoop directory /contrib/eclipse-plugin/hadoop-0.20.2-eclipse-plugin.jar
Everything seems fine until I try to run the application, and I see that I cannot run the application on Hadoop. You can click on Run on Hadoop and nothing happens.
Plugin version : 0.20.2
Hadoop version : 0.20.2
So then I find that this error was fixed later, I downloaded hadoop 0.20.203 and copied over its plugin from /contrib/eclipse-plugin/hadoop-eclipse-plugin-0.20.203.0.jar into eclipse plugin directory and started eclipse
eclipse -clean
Plugin version : 0.20.203.0
Hadoop version : 0.20.2
Now I can see that I can run my application.
But I have another issue I cannot look into DFS location! WTH Hadoop!
I get a new error : Connecting to DFS has encountered a problem. An internal error has occurred during “Connecting to DFS localhost. org/apache/commons/configuration/Configuration”
Hadoop 0.20.203.0 plugin for Eclipse fixes one problem and introduces a new one, that I can run the application on hadoop, but I cannot see the HDFS file system through plugin. So in the end I resorted to using these commands to view files on HDFS file system.
// to view the files in the directory hadoop fs -ls // to delete files from a directory hadoop fs -rm myfolder/sub2/* // to delete empty directory hadoop fs -rm myfolder/sub2 // to delete recursively hadoop fs -rmr myfolder/sub2 // copy file sample.txt to HDFS from current directory in native file system hadoop fs -put sample.txt . // get file sample.txt from HDFS to current directory in native file system hadoop fs -get sample.txt .
Later I managed to run my mapreduce program on command line, you can find the notes posted here.
[...] few days ago I had problems with my hadoop eclipse plugin, so I had to use command line to run my hadoop programs. If you also need to run your mapreduce [...]