Skip to content

Fix JobSubmission.getStagingDir incompatiblity #84

@rajubairishetti

Description

@rajubairishetti

while running with YARN client jars:
28 Jul 2014 13:54:01,027 [MergedStreamService_ss1@_sharad_sharad] WARN com.inmobi.conduit.distcp.MergedStreamService - Error in distcp
java.lang.NoSuchMethodError: org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(Lorg/apache/hadoop/mapred/JobClient;Lorg/apache/hadoop/conf/Configuration;)Lorg/apache/hadoop/fs/Path;
at org.apache.hadoop.tools.DistCp.createMetaFolderPath(DistCp.java:350)
at org.apache.hadoop.tools.DistCp.(DistCp.java:77)
at com.inmobi.conduit.distcp.ConduitDistCp.(ConduitDistCp.java:57)
at com.inmobi.conduit.distcp.DistcpBaseService.executeDistCp(DistcpBaseService.java:123)
at com.inmobi.conduit.distcp.MergedStreamService.execute(MergedStreamService.java:134)
at com.inmobi.conduit.AbstractService.run(AbstractService.java:162)
at java.lang.Thread.run(Thread.java:695)

The distcp code needs to be modified to do similar to this:
https://github.com/InMobi/DistCpV2-0.20.203/blob/master/src/main/java/org/apache/hadoop/tools/DistCp.java#L354

Metadata

Metadata

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions