GitHub - fengs/hadoop-book: Example source code accompanying O'Reilly's "Hadoop: The Definitive Guide" by Tom White

fengs / hadoop-book Public

forked from tomwhite/hadoop-book

Notifications You must be signed in to change notification settings
Fork 0
Star 0

Example source code accompanying O'Reilly's "Hadoop: The Definitive Guide" by Tom White

www.hadoopbook.com/

0 stars 2.6k forks Branches Tags Activity

Star

Notifications

Branches Tags

Name		Name	Last commit message	Last commit date
Latest commit History 158 Commits
app3/src/main/sh		app3/src/main/sh
book		book
ch02		ch02
ch03		ch03
ch04-avro		ch04-avro
ch04		ch04
ch05		ch05
ch07		ch07
ch08		ch08
ch09/src/main		ch09/src/main
ch11		ch11
ch12		ch12
ch13		ch13
ch14		ch14
ch15		ch15
ch16		ch16
common		common
experimental		experimental
hadoop-examples		hadoop-examples
hadoop-meta		hadoop-meta
input		input
snippet		snippet
.gitignore		.gitignore
README		README
pom.xml		pom.xml

Repository files navigation

Example code for "Hadoop: The Definitive Guide, Third Edition" by Tom White.
Copyright (C) 2011 Tom White, 978-1-449-31152-0

http://www.hadoopbook.com/
http://oreilly.com/catalog/9781449311520/

The code is hosted at http://github.com/tomwhite/hadoop-book/. You can find code
for the first edition at http://github.com/tomwhite/hadoop-book/tree/1e, and
for the second edition at http://github.com/tomwhite/hadoop-book/tree/2e.

This version of the code has been tested with:
 * Hadoop 1.2.1/0.22.0/0.23.x/2.2.0
 * Avro 1.5.4
 * Pig 0.9.1
 * Hive 0.8.0
 * HBase 0.90.4/0.94.15
 * ZooKeeper 3.4.2
 * Sqoop 1.4.0-incubating
 * MRUnit 0.8.0-incubating

Before running the examples you need to install Hadoop, Pig, Hive, HBase,
ZooKeeper, and Sqoop (as appropriate) as explained in the book.

You also need to install Maven.

Then you can build the code with:

% mvn package -DskipTests

By default Hadoop 1.2.1 is used. This can be changed by specifying the
hadoop.version property, e.g.

% mvn package -DskipTests -Dhadoop.version=1.2.0

There are profiles for different Hadoop major versions and distributions,
specified in hadoop-meta/pom.xml, and they are specified using the hadoop.distro
property. For example, to use the default version of Hadoop 2:

% mvn package -DskipTests -Dhadoop.distro=apache-2

Again, you can specify hadoop.version to use a particular Hadoop 2 version:

% mvn package -DskipTests -Dhadoop.distro=apache-2 -Dhadoop.version=2.1.1-beta

You should then be able to run the examples from the book.

Chapter names for "Hadoop: The Definitive Guide", Third Edition

ch01 - Meet Hadoop
ch02 - MapReduce
ch03 - The Hadoop Distributed Filesystem
ch04 - Hadoop I/O
ch05 - Developing a MapReduce Application
ch06 - How MapReduce Works
ch07 - MapReduce Types and Formats
ch08 - MapReduce Features
ch09 - Setting Up a Hadoop Cluster
ch10 - Administering Hadoop
ch11 - Pig
ch12 - Hive
ch13 - HBase
ch14 - ZooKeeper
ch15 - Sqoop
ch16 - Case Studies

app1 - Installing Apache Hadoop
app2 - Cloudera's Distribution for Hadoop
app3 - Preparing the NCDC Weather Data