cloud computing: Hive

Hive is a data warehouse system for Hadoop.
 Hive provides a SQL-like language called HiveQL. Due its SQL-like interface, Hive is increasingly becoming the technology of choice for using Hadoop

Prerequisites

The following are the prerequisites for setting up Hive and running Hive queries
  • You should have the latest stable build of Hadoop
  • To install hadoop,
  • Your machine should have Java 1.6 installed
  • It is assumed you have some knowledge of Java programming and are familiar with concepts such as classes and objects, inheritance, and interfaces/abstract classes.
  • Basic knowledge of Linux will help you understand many of the linux commands used in the tutorial

Setting up Hive

Platform

This tutorial assumes Linux. If using Windows, please install Cygwin. It is required for shell support in addition to the required software above.

Procedure

Download the most recent stable release of Hive as a tarball from one of the apache download mirrors. For our tutorial, we are going to use hive-0.9.0.tar.gz
Unpack the tarball in the directory of your choice, using the following command 
  $ tar -xzvf hive-x.y.z.tar.gz  
Set the environment variable HIVE_HOME to point to the installation directory:
You can either do
  $ cd hive-x.y.z
  $ export HIVE_HOME={{pwd}}
 
or set HIVE_HOME in $HOME/.profile so it will be set every time you login.
Add the following line to it.
  export HIVE_HOME=<path_to_hive_home_directory>
e.g.
  export HIVE_HOME='/Users/Work/hive-0.9.0'
  export PATH=$HADOOP_HOME/bin:$HIVE_HOME/bin:$PATH
Start Hadoop (Refer to the Single-Node Hadoop Setup Guide for more information). It should show the processes being started. You can check the processes started by using the jps query
$ start-all.sh
<< Starting various hadoop processes >>
$ jps
  3097 Jps
  2355 RunJar
  2984 JobTracker
  2919 SecondaryNameNode
  2831 DataNode
  2743 NameNode
  3075 TaskTracker
In addition, you must create /tmp and /user/hive/warehouse (aka hive.metastore.warehouse.dir) and set aprpopriate permissions in HDFS before a table can be created in Hive as shown below:
  $ hadoop fs -mkdir /tmp
  $ hadoop fs -mkdir /user/hive/warehouse
  $ hadoop fs -chmod g+w /tmp
  $ hadoop fs -chmod g+w /user/hive/warehouse

Comments

Post a Comment

Popular posts from this blog

Installing Qt Jambi For java

Installing from a Prebuilt Package After extracting the package, you are ready to run the Launcher, which is an application that runs the examples and demos that come with Qt Jambi. To run the examples and demo launcher on Windows run: qtjambi.exe On Linux and Mac OS X run: qtjambi.sh Running these files will verify the virtual machine version and set the correct environment for running Qt Jambi applications. When you are ready to distribute your own applications, see the  deployment  document. Manually Launching Qt Jambi Applications To manually launch applications, you need to include the following  .jar  files in your classpath: qtjambi-4.4.0_01.jar , which contains the Jambi classes. The platform archive, which contains the native libraries used by Qt Jambi. The archive is named after the operating system and compiler used to build the libraries; for instance,  qtjambi-win32-msvc2005-4.4.0_01.jar  for Windows and MSVC 2005. To run the examples or launcher, the class
Read more

Leach protocol installation in network simulator 2

Leach protocol installation in ns2 had to go through the following steps:  As far as I have studied, I got two patches for Leach 1. MIT patch 2. Mannasim patch- I didnt work much with it but if you dont want to meddle much with tcl, you can use it. Ok, the prerequisite is Ubuntu 10.04. We need to install ns-2.34  over that because LEACH MIT patch was made in 2000. Its a little painful to use the old OS, but it has to be done. Again you can have your OS on a virtual box on windows (only if you are not concerned with speed and not in love with your computer). 1. Download  ns-234-leach.tar.gz. Keep this in your /home/$userID/ns-allinone-2.34/ns-2.34. (Assuming you ns-allinone is in home dir) Leach ns2.34   2. Download the bash file "leach-setup.sh" into the directory " /home/$userID/ ns-allinone-2.34/ns-2.34".   leach-sh 3- Now find /home/arun/ns-allinone-2.35/ns-2.35 in the leach-setup.sh file and replace it with your own path. /home/$username/ns-a
Read more

What Is Tomcat Default Administrator Password ?

By default, Tomcat does not enable admin or manager access. To enable it, you have to edit the “ %TOMCAT_FOLDER%/conf/tomcat-users.xml ” manually. File : tomcat-users.xml (before update) , initially, Tomcat comments all users and roles like above. <?xml version = '1.0' encoding = 'utf-8' ?> <tomcat-users> <!-- <role rolename="tomcat"/> <role rolename="role1"/> <user username="tomcat" password="tomcat" roles="tomcat"/> <user username="both" password="tomcat" roles="tomcat,role1"/> <user username="role1" password="tomcat" roles="role1"/> --> </tomcat-users> File : tomcat-users.xml (after updated) <?xml version = '1.0' encoding = 'utf-8' ?> <tomcat-users> <!-- <role rolename="tomcat"/> <role rolename="role1"
Read more