cloud computing: Hive

Hive is a data warehouse system for Hadoop.
 Hive provides a SQL-like language called HiveQL. Due its SQL-like interface, Hive is increasingly becoming the technology of choice for using Hadoop

Prerequisites

The following are the prerequisites for setting up Hive and running Hive queries
  • You should have the latest stable build of Hadoop
  • To install hadoop,
  • Your machine should have Java 1.6 installed
  • It is assumed you have some knowledge of Java programming and are familiar with concepts such as classes and objects, inheritance, and interfaces/abstract classes.
  • Basic knowledge of Linux will help you understand many of the linux commands used in the tutorial

Setting up Hive

Platform

This tutorial assumes Linux. If using Windows, please install Cygwin. It is required for shell support in addition to the required software above.

Procedure

Download the most recent stable release of Hive as a tarball from one of the apache download mirrors. For our tutorial, we are going to use hive-0.9.0.tar.gz
Unpack the tarball in the directory of your choice, using the following command 
  $ tar -xzvf hive-x.y.z.tar.gz  
Set the environment variable HIVE_HOME to point to the installation directory:
You can either do
  $ cd hive-x.y.z
  $ export HIVE_HOME={{pwd}}
 
or set HIVE_HOME in $HOME/.profile so it will be set every time you login.
Add the following line to it.
  export HIVE_HOME=<path_to_hive_home_directory>
e.g.
  export HIVE_HOME='/Users/Work/hive-0.9.0'
  export PATH=$HADOOP_HOME/bin:$HIVE_HOME/bin:$PATH
Start Hadoop (Refer to the Single-Node Hadoop Setup Guide for more information). It should show the processes being started. You can check the processes started by using the jps query
$ start-all.sh
<< Starting various hadoop processes >>
$ jps
  3097 Jps
  2355 RunJar
  2984 JobTracker
  2919 SecondaryNameNode
  2831 DataNode
  2743 NameNode
  3075 TaskTracker
In addition, you must create /tmp and /user/hive/warehouse (aka hive.metastore.warehouse.dir) and set aprpopriate permissions in HDFS before a table can be created in Hive as shown below:
  $ hadoop fs -mkdir /tmp
  $ hadoop fs -mkdir /user/hive/warehouse
  $ hadoop fs -chmod g+w /tmp
  $ hadoop fs -chmod g+w /user/hive/warehouse

Comments

Post a Comment

Popular posts from this blog

Installing Qt Jambi For java

Installing from a Prebuilt Package After extracting the package, you are ready to run the Launcher, which is an application that runs the examples and demos that come with Qt Jambi. To run the examples and demo launcher on Windows run: qtjambi.exe On Linux and Mac OS X run: qtjambi.sh Running these files will verify the virtual machine version and set the correct environment for running Qt Jambi applications. When you are ready to distribute your own applications, see the  deployment  document. Manually Launching Qt Jambi Applications To manually launch applications, you need to include the following  .jar  files in your classpath: qtjambi-4.4.0_01.jar , which contains the Jambi classes. The platform archive, which contains the native libraries used by Qt Jambi. The archive is named after the operating system and compiler used to build the libraries; for instance,  qtjambi-win32-msvc2005-4.4.0_01.jar  for Windows and MSVC 2005. To run the ex...
Read more

What Is Tomcat Default Administrator Password ?

By default, Tomcat does not enable admin or manager access. To enable it, you have to edit the “ %TOMCAT_FOLDER%/conf/tomcat-users.xml ” manually. File : tomcat-users.xml (before update) , initially, Tomcat comments all users and roles like above. <?xml version = '1.0' encoding = 'utf-8' ?> <tomcat-users> <!-- <role rolename="tomcat"/> <role rolename="role1"/> <user username="tomcat" password="tomcat" roles="tomcat"/> <user username="both" password="tomcat" roles="tomcat,role1"/> <user username="role1" password="tomcat" roles="role1"/> --> </tomcat-users> File : tomcat-users.xml (after updated) <?xml version = '1.0' encoding = 'utf-8' ?> <tomcat-users> <!-- <role rolename="tomcat"/> <role rolename="role1"...
Read more

Install NS3

Install NS3 (ns-allinone-3.17) on Ubuntu 12.04/14.04 for beginners Installation steps for NS3 (ns-allinone-3.17): Follow the given steps for the installation of NS-3. First install some dependencies to enable the NS-3 execution environment on your Ubuntu 12.04/14.04 LTS machine. sudo apt-get install gcc g++ python sudo apt-get install gcc g++ python python-dev sudo apt-get install mercurial sudo apt-get install bzr sudo apt-get install gdb valgrind sudo apt-get install gsl-bin libgsl0-dev libgsl0ldbl sudo apt-get install flex bison libfl-dev sudo apt-get install g++-3.4 gcc-3.4 sudo apt-get install tcpdump sudo apt-get install sqlite sqlite3 libsqlite3-dev sudo apt-get install libxml2 libxml2-dev sudo apt-get install libgtk2.0-0 libgtk2.0-dev sudo apt-get install vtun lxc sudo apt-get install uncrustify sudo apt-get install doxygen graphviz imagemagick sudo apt-get install texlive texlive-extra-utils texlive-latex-extra sudo apt-get install python-s...
Read more