All Questions
Tagged with database-partitioning hive
23 questions
1
vote
1
answer
849
views
Pyspark updating a particular partition of an external Hive table
I am trying to overwrite a particular partition of a hive table using pyspark but each time i am trying to do that, all the other partitions are getting wiped off. I went through couple of posts in ...
- 131
0
votes
1
answer
317
views
Change column name of an external partitioned parquet table in hive without null/lost data
I have the following table:
CREATE EXTERNAL TABLE aggregate_status(
m_point VARCHAR(50),
territory VARCHAR(50),
reading_meter VARCHAR(50),
meter_type VARCHAR(500)
)
PARTITIONED BY(
insert_date VARCHAR(...
- 13
0
votes
1
answer
2k
views
Hive, how to partition by a colum with null values, putting all nulls in one partition
I am using Hive, and the IDE is Hue. I am trying different key combinations to choose for my partition key(s).
The definition of my original table is as follows:
CREATE External Table `my_hive_db`.`...
- 1,421
0
votes
1
answer
1k
views
Hive, how to drop partition, Error while compiling statement: FAILED: expecting set null in drop partition statement
I have a table in Hive which I would like to drop its partition keys for later using other partition keys.
The location of parquet file is in Amazon S3. The table, I am working on, is partitioned by ...
- 1,421
0
votes
1
answer
392
views
Error inserting data into Hive partitioned table
I'm trying to insert data into a Hive table with partition, the partition condition is yesterday's date in yyyyMMdd format, and I want to do that dynamically so I'm generating it using a query. The ...
- 3
0
votes
1
answer
72
views
Hive Dyanamic Partitioning failing to write partitions to final subdirectories
I am still new to hive
I have the following table loaded with data
CREATE TABLE table1 (
col1 string,
col2 string,
col3 float,
col4 float)
ROW FORMAT DELIMITED FIELDS TERMINATED BY ',';
and ...
1
vote
1
answer
664
views
Hive Date Partitioned table - Streaming Data in S3 with mixed dates
I have extensive experience working with Hive Partitioned tables. I use Hive 2.X. I was interviewing for a Big Data Solution Architect role and I was asked the below question.
Question: How would ...
- 135
0
votes
1
answer
377
views
Can't overwrite partition in Hive table
I'm trying to overwrite a certain partition of a table. My query is pretty large, but I managed to find a minimal code that reproduces the error. When I try to run this:
INSERT OVERWRITE TABLE sbx....
- 1,166
0
votes
1
answer
286
views
Delete partition with non-constant value in Hive
I want to delete a partition in Hive with its value being in another table or being created by a function on-the-fly. For example:
ALTER TABLE
table_1
DROP IF EXISTS
PARTITION (dt = ...
- 1,166
1
vote
1
answer
1k
views
Hive - Copy database schema with partitions and recreate in another hive instance
I have copied the data and folder structure for a database with partitioned hive tables from one HDFS instance to another.
How can I do the same with the hive metadata? I need the new HDFS instance'...
- 39.4k
0
votes
0
answers
36
views
Create Partition Based on Customer Account
I need to partition a big Hive Table based on customer identifier (account number like 12345678).
In particular, I am looking for a function that gets customer_id and then converts it into a ...
- 1,150
0
votes
0
answers
156
views
Hive: Finding average time-difference between rows per group
I have a table with rows consisting of in hdfs.
I want to compute the average difference in date for each id for the table.
This is the query I am using , but it keeps giving me error:-
select id, ...
- 57
0
votes
1
answer
2k
views
Identifying Partitioned tables in Hive [duplicate]
Is there any way which allows listing of partitioned tables in Hive?
I found the way which allows this to happen in SQL Server.
https://dba.stackexchange.com/questions/14996/how-do-i-get-a-list-of-...
- 1,097
4
votes
1
answer
1k
views
How to disable autocommit in JDBC using clojure
I want to use Clojure's JDBC interface to update an Hive database. In particular, I want to add partitions. The code would be like the following:
(jdbc/db-do-commands hive-db ["ALTER TABLE foo ADD ...
- 1,886
0
votes
1
answer
1k
views
How to load multiple CSV tables recursively in one directory into Hive
I have created an external Hive table with specified schema but without data, say table A. Now let us say I have CSV files in HDFS dir organized in the following way:
20150718/dir1/dir2/file1.csv
...
- 1,223