Loading Data From Local File System to Hive

Importing data into an RDBMS is a separate feature or function, although it is one of the DML commands in Hive. Data may be imported into a Hive table from HDFS or a local system. We’ll speak about importing data from local file system into Hive in this post.

Syntax:
LOAD DATA LOCAL INPATH <HDFS-Location> OVERWRITE INTO TABLE <Table-Name>

The use of “OVERWRITE” is optional, and it is intended to overwrite existing data. If the ‘OVERWRITE’ keyword is omitted, data files are appended to existing data sets. The load command does not do any data validation against the schema. The file is transferred into the Hive-controlled file system namespace if it is in HDFS. The input file is on the local file system, as indicated by the ‘LOCAL’ suffix. If ‘LOCAL’ is not specified, the file is searched in HDFS.

Examples:
LOAD DATA LOCAL INPATH ‘/user/cloudera/testfolder/test.txt’
INTO TABLE TestTable;
The command above will import data from ‘test.txt’ into an existing Hive table named “TestTable.”

LOAD DATA LOCAL INPATH ‘/user/cloudera/testfolder/JulySales.csv’
OVERWRITE INTO TABLE tblSales PARTITION (month=’July’);
The command above will import data from “JulySales.csv” into a partitioned Hive table called “tblSales.” Because overwrite is utilized, the existing data will be replaced.

Hope you find this article helpful.

Please subscribe for more interesting updates.

4 comments

  1. When you implement a big data processing scenario, the first step is to load the data into Hadoop. The data source is typically in the local file system, HDFS, Hive tables, or external Hive tables. After the data is loaded, you can validate and transform it by using HiveQL like you use SQL. You can perform data validation (such as checking for NULLS and primary keys), and transformations (such as filtering, aggregations, set operations, and derived tables). You can also include customized procedural snippets (scripts) for processing the data.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s