Hadoop Hangover: Introduction To Apache Bigtop and Installing Hive, HBase and Pig

In the previous post we learnt how easy it was to install Hadoop with Apache Bigtop!
We know its not just Hadoop and there are sub-projects around the table! So, lets have a look at how to install Hive, Hbase and Pig in this post.


Before rowing your boat...
Please follow the previous post and get ready with Hadoop installed!
Follow the link for previous post:
http://femgeekz.blogspot.in/2012/06/hadoop-hangover-introduction-to-apache.html
also, the same can be found at DZone, developer site: http://www.dzone.com/links/hadoop_hangover_introduction_to_apache_bigtop_and.html


All Set?? Great! Head On..
Make sure all the services of Hadoop are running. Namely, JobTracker, SecondaryNameNode, TaskTracker, DataNode and NameNode. [standalone mode]


Hive with Bigtop:
The steps here are almost the same as Installing Hive as a separate project.
However, few steps are reduced.
The Hadoop installed in the previous post is Release 1.0.1


We had installed Hadoop with the following command
sudo apt-get install hadoop\*
Step 1: Installing Hive
We have installed Bigtop 0.3.0, and so issuing the following command installs all the hive components.
ie. hive, hive-metastore, hive-server. The daemons names are different in Bigtop 0.3.0.
sudo apt-get install hive\*
This installs all the hive components. After installing, the scripts must be able to create /tmp and /usr/hive/warehouseand HDFS doesn't allow these to be created while installing as it is unaware of the path to Java. So, create the directories if not created and grant the execute permissions.
In the hadoop directory, ie. /usr/lib/hadoop/
bin/hadoop fs -mkdir /tmp
bin/hadoop fs -mkdir /user/hive/warehouse
bin/hadoop -chmod g+x /tmp
bin/hadoop -chmod g+x /user/hive/warehouse


Step 2: The alternative directories could be/var/run/hiveand/var/lock/subsys
sudo mkdir /var/run/hive
sudo mkdir /var/lock/subsys


Step 3: Start the hive server, a daemon
sudo /etc/init.d/hive-server start
Image:

start hive-server

Step 4: Running Hive
Go-to the directory /usr/lib/hive.
See the Image below:
bin/hive

bin/hive

Step 5: Operations on Hive
Image: 

Basic hive operations

HBase with Bigtop:
Installing Hbase is similar to Hive.


Step 1: Installing HBase
sudo apt-get install hbase\*
Image: 

hbase-0.92.0

Step 2: Starting HMaster
sudo service hbase-master start
Image: 

Starting HMaster

Image: 

jps (HMaster started)

Step 3: Starting HBase shell
hbase shell
Image: 

start HBase shell

Step 4: HBase Operations
Image: 

HBase table operations

Image: 

list,scan,get,describe In HBase

Pig with Bigtop:
Installing Pig is similar too.


Step 1: Installing Pig
sudo apt-get install pig
Image: 

Installing Pig

Step 2: Moving a file to HDFS
Image: 

Moving a tab separated file "book.csv" to HDFS

Step 3: Installed Pig-0.9.2
Image: 

Pig installed Pig-0.9.2

Step 4: Starting the grunt shell
pig
Image: 

Starting Pig

Step 5: Pig Basic Operations
Image:

Basic Pig Operations

Image:

Job Completion

We saw that is it possible to install the subprojects and work with Hadoop, with no issues.
Apache Bigtop has its own spark! :)
There is a release coming BIGTOP-0.4.0 which is supposedly to fix the following issues:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12318889&styleName=Html&projectId=12311420
Source and binary files:
http://people.apache.org/~rvs/bigtop-0.4.0-incubating-RC0
Maven staging repo:
https://repository.apache.org/content/repositories/orgapachebigtop-279
Bigtop's KEYS file containing PGP keys we use to sign the release:
http://svn.apache.org/repos/asf/incubator/bigtop/dist/KEYS


Let us see how to install other sub-projects in the coming posts!
Until then, Happy Learning!! :):)

Say Hello To ANDROID: [Part I] Introduction

    Its an era of mobile, tablets and a very exciting time for developers!! Mobile phones have never been more popular and powerful. Smart Phones have become very stylish, versatile packing hardware features like GPS, accelerometers, etc. are an enticing platform which kindles the developers to create some innovative mobile applications.

    With the existing mobile development built on proprietary operating systems that restrict third-party applications, Android offers an open and the best alternative. Without artificial barriers, Android developers are free to write applications that take full advantage of increasingly powerful mobile hardware. As a result, developer interest in Android devices has made their 2008 release a hugely anticipated mobile technology event.

    The open philosophy of android is most welcomed by majority of the developers as it features very powerful SDK libraries. Experienced mobile developers can now tinker with android and explore the platform, leveraging the unique features to enhance existing products or create more innovative ones.

ANDROID
    Its an open source software stack that includes
• Operating System
• Middle ware
• Key Applications + set of API Libraries
which changes the look, feel and function of the mobile.

    In Android, native and third-party applications are written using the same APIs and executed on the same run time. These APIs feature hardware access, location-based services, support for background services, map-based activities, relational databases, inter-device peer-to-peer messaging and 2D and 3D graphics.

Just A Flashback ...

    Historically, the developers had to code in low level C or C++ and had to learn the specific hardware feature upon which they were coding. But, as the hardware features enhanced this became more cumbersome. More recently, Symbian was far successful in giving a room for the developers to better leverage the hardware available. However, it required writing complex C/C++ code and making heavy use of proprietary APIs that are notoriously difficult to use. This difficulty was amplified when developing applications that must work on different hardware implementations and  particularly true when developing for a particular hardware feature like GPS.

    Then came the Java hosted MIDIlets that are executed on the same JVM, abstracting the underlying hardware and letting the developers create apps that run on wide variety of hardware that supports Java run time.

    So our Andy is here... :) Ouch! forgot about Apple's iPhone, Windows mobiles ?? ;) No offence!
    They provide richer UI, UX and simplified development environment! "BUT" they’re built on proprietary operating systems that often prioritize native applications over those created by third parties and restrict communication among applications and native phone data.

    Third-party and native Android applications are written using the same APIs and are executed on the same run time. Users can remove and replace any native application with a third-party developer alternative; even the dialer and home screens can be replaced...{ Wanna Try ?? :p }

    Google describes Android as:
"The first truly open and comprehensive platform for mobile devices, all of the software to run a mobile phone but without the proprietary obstacles that have hindered mobile innovation."
          http://googleblog.blogspot.com/2007/11/wheres-my-gphone.html   :)

    The FLASHBACK is incomplete...... without the introduction of OHA { Oh Haa ??  ;) }

OHA: Open Handset Alliance comprises of
• Developers
• Hardware Manufacturers
• Mobile Carriers

The tech companies involved prominently are Motorola, HTC, T-Mobile, Qualcomm and their words:
"A commitment to openness, a shared vision for the future, and concrete plans to make the vision a reality. To accelerate innovation in mobile and offer consumers a richer, less expensive, and better
mobile experience."
                http://www.openhandsetalliance.com/oha_faq.html :)

        Android offers an excellent enterprise platform, has targeted developers making their lives far simple with Google and the OHA betting that the way to deliver
better mobile software to consumers is by making it easier for developers to write it themselves. This openness and power ensure that anyone with the inclination can bring a vision to life at minimal cost.

       Use Open source softwares, share them and give back something better to the community..... !! :)

       Next post...
• Merits of android
• unboxing SDK features, Software stack and lots more ..... :)


      Well, http://developer.android.com is a great reference site!
                                   
                                              Happy Learning! :)