Sqoop Complete Tutorial Part-13

This is continuation part to “SQOOP Complete Tutorial” and in this part we will see how to create and save the jobs in Sqoop.

Instead of rewriting commands that need to be run many times, Sqoop allows you to save them as jobs. When it comes to incremental import, this is a common occurrence. A saved job keeps track of the configuration information needed to run a Sqoop operation later. Job descriptions are stored to a private repository at $HOME/.sqoop/ by default, but this can be modified to make saved jobs available to many users on a shared cluster.

The following are the arguments that you should be aware of while dealing with Sqoop jobs.

–create <job-id>
Define a new saved job with the specified job-id (name). A second Sqoop command-line, separated by a — should be specified; this defines the saved job.

Examples:
      sqoop job
          –create JobTestDBImport
          — import
          –connect jdbc:mysql://localhost/NewTestDB
          –username root
          –password cloudera
          –table emp
          –target-dir /user/cloudera/emp

The user credentials can be ignore while creating the job and can be passed during the execution.

sqoop job
–create JobTestDBImport
— import
–connect jdbc:mysql://localhost/NewTestDB
–table emp
–target-dir /user/cloudera/emp

     sqoop job
       –create JobTestDBExport
       –export
       –connect jdbc:mysql://localhost/NewTestDB
       –username root
       –password cloudera
       –table Emp
       –export-dir /user/cloudera/empdir
       –update-key EmpID
       –update-mode allowinsert

–delete <job-id>
Delete a saved job.

    Examples:
$ sqoop job –delete JobTestDBImport

–exec <job-id>
Given a job defined with –create, run the saved job.

    Examples:
$ sqoop job –exec JobTestDBImport

    $ sqoop job –exec JobTestDBImport –username root -P
    Enter password:

–show <job-id>
Show the parameters for a saved job.

    Example:
    $ sqoop job –show JobTestDBExport

–list
List all saved jobs

Example:

     $ sqoop job –list

Hope you find this article helpful.

Please subscribe for more interesting updates.

2 comments

  1. I’m extremely inspired together with your writing skills as well as with the structure in your blog.
    Is this a paid subject or did you customize
    it your self? Anyway keep up the nice high quality writing,
    it is rare to look a nice weblog like this one these days..

    Like

  2. Hey! Someone in my Myspace group shared this website with us
    so I came to take a look. I’m definitely loving the information. I’m bookmarking and will be tweeting this to my followers!
    Superb blog and wonderful design.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s