2011/06/19

how to make jar file for custom Hive UDF

I tried to make my original Hive User Defined Function (Hive UDF). But it was bit difficult for me to compile java file correctly, because I was a beginner of JAVA and did not know much about JAVA compile options. For compiling JAVA file and making JAR file for Hive UDF, I implemented a shell script which is almost totally copied from here 8^)

script for compiling JAVA file and making JAR file


This code is written by sh script. If you want to use bash version, you can get source code from here

#!/bin/sh

if [ $# -ne 1 ]; then
echo "Usage: $0 "
exit 1
fi

CNAME=${1%.java}
JARNAME=$CNAME.jar
JARDIR=/home/yaboo/$CNAME
CLASSPATH=$(ls $HIVE_HOME/lib/hive-serde-*.jar):$(ls $HIVE_HOME/lib/hive-exec-*.jar):$(ls $HADOOP_HOME/hadoop-core-*.jar)
echo "javac -classpath $CLASSPATH -d $JARDIR/ $1 && jar -cf $JARNAME -C $JARDIR/ . && tell $1"

tell() {
echo
echo "$1 successfully compiled."
echo
}

mkdir -p $JARDIR
javac -classpath $CLASSPATH -d $JARDIR/ $1 && jar -cf $JARNAME -C $JARDIR/ . && tell $1


custom UDF


Here, I show custom UDF named Lower. This code is copied from official Hive Wiki

package com.example.hive.udf;

import org.apache.hadoop.hive.ql.exec.UDF;
import org.apache.hadoop.io.Text;

public final class Lower extends UDF {
public Text evaluate(final Text s) {
if (s == null) {
return null;
}
return new Text (s.toString().toLowerCase());
}
}


Usage of script


> sh compile.sh Lower.java
Lower.java successfully compiled.
> ls Lower*
Lower.jar Lower.java Lower

No comments:

Post a Comment

100