导航：首页 > 服务器 >

Hadoop怎么自定制数据类型

发表于：2025-01-31 作者：千家信息网编辑

千家信息网最后更新 2025年01月31日，这篇文章主要讲解了"Hadoop怎么自定制数据类型"，文中的讲解内容简单清晰，易于学习与理解，下面请大家跟着小编的思路慢慢深入，一起来研究和学习"Hadoop怎么自定制数据类型"吧！一般有两个办法，一

千家信息网最后更新 2025年01月31日Hadoop怎么自定制数据类型

这篇文章主要讲解了"Hadoop怎么自定制数据类型"，文中的讲解内容简单清晰，易于学习与理解，下面请大家跟着小编的思路慢慢深入，一起来研究和学习"Hadoop怎么自定制数据类型"吧！

一般有两个办法，一种较为简单的是针对值，另外一种更为完整的是对于键和值都适应的方法：
1、实现Writable接口：

/* DataInput and DataOutput 类是java.io的类 */
public interface Writable {
void readFields(DataInput in);
void write(DataOutput out);
}

下面是一个小例子：
public class Point3D implement Writable {
public float x, y, z;
public Point3D(float fx, float fy, float fz) {
this.x = fx;
this.y = fy;
this.z = fz;
}
public Point3D() {
this(0.0f, 0.0f, 0.0f);
}
public void readFields(DataInput in) throws IOException {
x = in.readFloat();
y = in.readFloat();
z = in.readFloat();
}
public void write(DataOutput out) throws IOException {
out.writeFloat(x);
out.writeFloat(y);
out.writeFloat(z);
}
public String toString() {
return Float.toString(x) + ", "
+ Float.toString(y) + ", "
+ Float.toString(z);
}
}

2、对于键来说，需要指定排序规则，对此，Java版Hadoop的办法是实现WritableComparable这个泛型接口，WritableComparable，顾名思义了，一半是Writable，一半是Comparable

public interface WritableComparable {
public void readFields(DataInput in);
public void write(DataOutput out);
public int compareTo(T other);
}

这里的compareTo方法是默认的key排序

先给出下面的简单例子，再做说明和扩展。
public class Point3D inplements WritableComparable {
public float x, y, z;
public Point3D(float fx, float fy, float fz) {
this.x = fx;
this.y = fy;
this.z = fz;
}
public Point3D() {
this(0.0f, 0.0f, 0.0f);
}
public void readFields(DataInput in) throws IOException {
x = in.readFloat();
y = in.readFloat();
z = in.readFloat();
}
public void write(DataOutput out) throws IOException {
out.writeFloat(x);
out.writeFloat(y);
out.writeFloat(z);
}
public String toString() {
return Float.toString(x) + ", "
+ Float.toString(y) + ", "
+ Float.toString(z);
}
public float distanceFromOrigin() {
return (float) Math.sqrt( x*x + y*y +z*z);
}
public int compareTo(Point3D other) {
return Float.compareTo(
distanceFromOrigin(),
other.distanceFromOrigin());
}
public boolean equals(Object o) {
if( !(o instanceof Point3D)) {
return false;
}
Point3D other = (Point3D) o;
return this.x == o.x
&& this.y == o.y
&& this.z == o.z;
}
/* 实现 hashCode() 方法很重要
* Hadoop的Partitioners会用到这个方法，后面再说
*/
public int hashCode() {
return Float.floatToIntBits(x)
^ Float.floatToIntBits(y)
^ Float.floatToIntBits(z);
}
}
如果要将对象写入数据库则还要继承DBWritable接口

public interface WritableComparable {
public void write(PreparedStatement statement) throwsSQLException;
public void readFields(ResultSet resultSet) throws SQLException;}

下面写个例子

public class LocationBean implements Writable, DBWritable {
private String mobilenetworkcode;

private String mobilecountrycode;

private Integer cellid;

private Integer locationareacode;

private Integer baiduareaid;

private Double lat;

private Double lng;

private Integer areaid;
@Override
public void write(PreparedStatement statement) throws SQLException {
int index = 1;
statement.setString(index++, this.getMobilenetworkcode());
statement.setString(index++, this.getMobilecountrycode());
statement.setInt(index++, this.getCellid());
statement.setInt(index++, this.getLocationareacode());
statement.setInt(index++, this.getBaiduareaid());
statement.setDouble(index++, this.getLat());
statement.setDouble(index++, this.getLng());
statement.setInt(index, this.getAreaid());
}

@Override
public void readFields(ResultSet resultSet) throws SQLException {
this.mobilenetworkcode = resultSet.getString(1);
this.mobilecountrycode = resultSet.getString(2);
this.cellid = resultSet.getInt(3);
this.locationareacode = resultSet.getInt(4);
this.baiduareaid = resultSet.getInt(5);
this.lat = resultSet.getDouble(6);
this.lng = resultSet.getDouble(7);
this.areaid = resultSet.getInt(8);
}

@Override
public void write(DataOutput out) throws IOException {
// TODO Auto-generated method stub

}

@Override
public void readFields(DataInput in) throws IOException {

}

public String getMobilenetworkcode() {
return mobilenetworkcode;
}

public void setMobilenetworkcode(String mobilenetworkcode) {
this.mobilenetworkcode = mobilenetworkcode;
}

public String getMobilecountrycode() {
return mobilecountrycode;
}

public void setMobilecountrycode(String mobilecountrycode) {
this.mobilecountrycode = mobilecountrycode;
}

public Integer getCellid() {
return cellid;
}

public void setCellid(Integer cellid) {
this.cellid = cellid;
}

public Integer getLocationareacode() {
return locationareacode;
}

public void setLocationareacode(Integer locationareacode) {
this.locationareacode = locationareacode;
}

public Integer getBaiduareaid() {
return baiduareaid;
}

public void setBaiduareaid(Integer baiduareaid) {
this.baiduareaid = baiduareaid;
}

public Double getLat() {
return lat;
}

public void setLat(Double lat) {
this.lat = lat;
}

public Double getLng() {
return lng;
}

public void setLng(Double lng) {
this.lng = lng;
}

public Integer getAreaid() {
return areaid;
}

public void setAreaid(Integer areaid) {
this.areaid = areaid;
}

}