Prash's Blog

Issues with Hadoop eclipse plugin February 2, 2012

Filed under: Hadoop — prazjain @ 10:17 am
Tags: , , ,

It has been a day that I am now playing with Hadoop and bumped into issues with Eclipse plugin for Hadoop (Later, I ran mapreduce program on command line, you can find the notes posted here).

I have Windows 7, 64 bit machine, eclipse Indigo 3.7.1, and installed hadoop 0.20.2 (why did I install 0.20.2).

Ok, and then I copied over the eclipse plugin from Hadoop directory /contrib/eclipse-plugin/hadoop-0.20.2-eclipse-plugin.jar

Everything seems fine until I try to run the application, and I see that I cannot run the application on Hadoop. You can click on Run on Hadoop and nothing happens.

Plugin version : 0.20.2

Hadoop version : 0.20.2

So then I find that this error was fixed later, I downloaded  hadoop 0.20.203 and copied over its plugin from /contrib/eclipse-plugin/hadoop-eclipse-plugin-0.20.203.0.jar  into eclipse plugin directory and started eclipse


eclipse -clean

Plugin version : 0.20.203.0

Hadoop version : 0.20.2

Now I can see that I can run my application.

But I have another issue I cannot look into DFS location! WTH Hadoop!

I get a new error : Connecting to DFS has encountered a problem. An internal error has occurred during “Connecting to DFS localhost. org/apache/commons/configuration/Configuration”

Hadoop 0.20.203.0 plugin for Eclipse  fixes one problem and introduces a new one, that I can run the application on hadoop, but I cannot see the HDFS file system through plugin. So in the end I resorted to using these commands to view files on HDFS file system.


// to view the files in the directory

hadoop fs -ls

// to delete files from a directory

hadoop fs -rm myfolder/sub2/*

// to delete empty directory

hadoop fs -rm myfolder/sub2

// to delete recursively

hadoop fs -rmr myfolder/sub2

// copy file sample.txt to HDFS from current directory in native file system

hadoop fs -put sample.txt .

// get file sample.txt from HDFS to current directory in native file system

hadoop fs -get sample.txt .

Later I managed to run my mapreduce program on command line, you can find the notes posted here.

Advertisements
 

5 Responses to “Issues with Hadoop eclipse plugin”

  1. […] few days ago I had problems with my hadoop eclipse plugin, so I had to use command line to run my hadoop programs. If you also need to run your mapreduce […]

  2. sunny Says:

    I need ur help .I stucked while configuring VMware server with eclipse.
    Actually,i am following YDN tutorial of haddop,they are asking for change two parameter value .
    1. hadoop.job.ugi 2. mapred.system.dir but first one is not there.so i am unable to manage that DFS Location hierarchy.
    I am using Eclipse europa 3.3 with hadoop 0.18.0.

  3. venu Says:

    I have hadoop 1.0.4 is it ok to use the plugin version 0,20.2 for running jobs on hadoop 1.0.4 .
    Please let me know


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s