1 / 18

Sqoop Installation Guide

Sqoop Installation Guide. Lecturer : Prof. Kyungbaek Kim Presenter : Zubair Amjad. Requirements. Hadoop Sqoop MySQL Server JDBC driver. Environment. Ubuntu 12.04 Hadoop 1.0.3 Sqoop 1.4.4. Hadoop Fully Distributed Mode. Sqoop Installation.

debbiep
Download Presentation

Sqoop Installation Guide

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Sqoop Installation Guide Lecturer : Prof. KyungbaekKim Presenter : ZubairAmjad

  2. Requirements • Hadoop • Sqoop • MySQL Server • JDBC driver

  3. Environment • Ubuntu 12.04 • Hadoop 1.0.3 • Sqoop1.4.4

  4. Hadoop Fully Distributed Mode

  5. Sqoop Installation • Download the sqoop-1.4.4.bin_hadoop-1.0.0.tar.gz file • ww.apache.org/dyn/closer.cgl/sqoop/1.4.4 • Unzip the tar file • sudotar -zxvfsqoop-1.4.4.bin_hadoop-1.0.0.tar.gz • Move sqoop-1.4.4.bin_hadoop-1.0.0 to sqoop • sudo mv sqoop1.4.4.bin_hadoop-1.0.0 sqoop • Create a directory sqoop in usr/lib • sudomkdir /usr/lib/sqoop • Go to the folder sqoopand copy files to /usr/lib/sqoop • sudo mv ./* /usr/lib/sqoop • Go to root directory and open bashrcfile • sudogedit ~/.bashrc • Add SQOOP_HOME variable • export SQOOP_HOME=/usr/lib/sqoop • export PATH=$PATH:$SQOOP_HOME/bin • Check sqoop installation • sqoop version

  6. MySQL Server • Install mysql server • sudo apt-get install mysql-server • Give appropriate username and password • Download mysql-connector-java-5.1.28-bin.jar and copy to /usr/lib/sqoop/lib • cp mysql-connnectpr-java-5.1.28-bin.jar /usr/lib/sqoop/lib/ • Login to mysql server • mysql -u root –p

  7. Create Database • Create a database • create database temp_db; • show databases; • Add a table in the database • use temp_db; • CREATE TABLE employee (id INT, name VARCHAR(20), email VARCHAR(20)); • Insert some rows using INSERT INTO • Grant privileges to root user • grant all privileges on *.* to root@IP IDENTIFIED BY ‘root’ WITH GRANT OPTION;

  8. Create Database

  9. Create Table

  10. Insert rows in table

  11. Grant Permissions

  12. Sqoop Import • Import the table from temp_db to HDFS • sqoop import -connect jdbc:mysql://localhost:3306/temp_db -username root -P -table employee –target-dir /sqoopOut –m 1

  13. Table Import

  14. HDFS

  15. Sqoop Export • Export data from HDFS to database table • Create a table in data base • CREATE TABLE stud1 (id INT, name VARCHAR(20)); • Export data in file stud to database • sqoopexport --connect jdbc:mysql://192.168.34.130/temp_db --username root -P --table stud1 -m 1 --export-dir /sqoopexport/stud

  16. Export

  17. Exported Table

  18. Thank you

More Related