26,697 questions
1
vote
0
answers
34
views
How to optimize special array_intersect in hive sql executed by spark engine?
buckets is a column of type array<string>. The logic is similar to array_intersect, except only the prefix of each string in buckets (before the first -) is compared. How can I optimize the ...
- 11
0
votes
0
answers
89
views
Flutter app stuck on launcher screen on some Android devices (Impeller / Firebase / AppLinks suspected)
I’m experiencing an issue where my Flutter app works fine on most devices, but on some Android devices and emulators, it gets stuck on the icon/logo screen during the first launch.
Details from my ...
0
votes
1
answer
53
views
Spark 3.3.1: decimal(38,10) shows as 0E-10 in DataFrame and inserts as NULL into Hive (Parquet / insertInto)
Environment:
Apache Spark: 3.3.1
Hive: 3.1.3 (metastore + Hive server)
Table storage format: PARQUET
Insertion method: dataframe.format("hive").insertInto("db.table")
Problem ...
1
vote
1
answer
80
views
Apache Hive Docker container: HiveServer2 fails to bind on port 10000 (Connection refused in Beeline
I am running Apache Hive 4.0.0 inside Docker on Ubuntu 22.04.
The container starts, but HiveServer2 never binds to the port.
When I try to connect with Beeline:
sudo docker exec -it hive4 beeline -u ...
- 11
0
votes
1
answer
118
views
Trino with Hive- Infer parquet schema
Background: I am running trino with Hive catalog that reads data from S3. Setup and querying through trino over these tables is working fine with table's metadata stored in hive metastore.
Question: I ...
- 75
0
votes
0
answers
153
views
spark 4.0 do not support hive 4.0.0
while I comiple spark 4.0 with hive 4.0.0, then I run into this issue. This looks like spark 4.0 is not compatable with hive 4.0.0. But the doc suggest 4.0.0 as one of the support. So I don't know ...
- 71
2
votes
0
answers
38
views
Count(*) query returns empty when using Tez, but works with MapReduce
I have hadoop + hive setup using docker, however when I try to run count(*) on my table it gives me an empty return when using Tez and the correct one when using MapReduce, the table is an external ...
- 21
0
votes
0
answers
58
views
Trino: USER ERROR — HIVE_EXCEEDED_SPLIT_BUFFERING_LIMIT
We are currently using Trino 457 version. This week we started updating Trino version. Starting from 459, we are facing a lot of
USER ERROR — HIVE_EXCEEDED_SPLIT_BUFFERING_LIMIT
io.trino.spi....
- 1,595
2
votes
1
answer
78
views
Merging SCD-2 intervals from two tables
I have two historical (SCD-2) tables, that should be merged in a new one as shown below, using HiveQL.
The earliest dates does match on both tables!
1st table
PK var1 start_dt end_dt
123 false ...
- 209
0
votes
1
answer
59
views
Unable to register database/table in aws glue when hudi job is submitted from emrserverless
I am using emr 6.15 and hudi 0.14
I submitted following hudi job which should create a database and a table in aws glue. IAM Role assigned to EMR serverless has all neccessary permissions of s3 and ...
- 294
0
votes
0
answers
59
views
How to set up Open Data Lakehouse using Spark, External Hive Metastore and S3?
I am trying to setup an Open Data Lakehouse for one of my personal projects where I have deployed Spark on my local setup. I also have Hive Metastore deployed using Docker which is using PostgreSQL ...
- 7
1
vote
0
answers
104
views
Title: Flutter build_runner "Nothing to build" but *.g.dart files are missing after pub get success
I'm developing a Flutter music app and using hive for local storage, which requires hive_generator and build_runner for code generation. I'm encountering a very persistent issue where build_runner ...
- 11
0
votes
1
answer
59
views
How does log4j clean up actually works in hive metastore?
I am trying to configure logs for hive metastore. I use this config:
status = INFO
name = HiveLog4j2
packages = org.apache.hadoop.hive.ql.log
property.hive.log.level = INFO
property.hive.root.logger =...
- 34k
0
votes
1
answer
133
views
Hive 4.0.1 doesn't work because of Jar files not found
Hive 4.0.1 doesn't work because of Jar files not found. I want to use hive integrated with hadoop 3.4.1 to query data on apache spark.
I tried to type in ./hive/bin/hive and expected it to return >...
0
votes
0
answers
237
views
How do I programmatically build a WITH 'ALIAS' AS (SELECT ...) CTE in sqlglot?
I'm trying to build a Common Table Expression (CTE) like this using the sqlglot Python library:
WITH A AS (
SELECT * FROM my_table
)
SELECT * FROM A
When I attempt to build this with the ...
