Skip to content

v2.0.0

Compare
Choose a tag to compare
@colinmarc colinmarc released this 05 Aug 21:52
· 128 commits to master since this release
fd1e410

This is a major release, including multiple breaking interface changes and new features.

The library is now structured as a go module. To use it, use the import path github.com/colinmarc/hdfs/v2.

Kerberos support

Added in #133, with lots of help from @Shastick and @staticmukesh. This adds basic kerberos authentication using to the library and the command line client - to the latter with support for ccaches. This much-requested feature should be ready for production use, but I would love your feedback and/or bug reports.

Timeouts and other useful configuration

Client now has two new options, NamenodeDialFunc and DatanodeDialFunc, which can be used to replace net.Dial and set timeouts and keepalives and other useful things. You can also use SetDeadline on FileReader and FileWriter to enforce i/o timeouts. See #139 for more information.

Your hdfs.Client can be made to respect your Hadoop configuration with the hdfs.ClientOptionsFromConf method. This looks for relevant options from the configuration and tries to configure the client to match. While this doesn't do that much right now, it may be expanded to other things in the future.

Incompatable interface changes

The rpc package is now internal; keeping the interface backwards compatible was too difficult, and nothing in that subpackage was really useful externally anyway. rpc.NamenodeError which was possibly the only useful export, now implements an interface, hdfs.Error.

The configuration parsing/loading code and the HadoopConf type have been moved into their own package, hadoopconf. The interfaces are also slightly tweaked.

Finally, all deprecated methods, such as hdfs.NewForUser, have been removed.