112,822 questions
2
votes
0
answers
79
views
Unexplained Scala compiler behavior (3.7.4)
I have posted a question related to "contains" in the past.
The signature of "contains" on Seq is:
def contains[A1 >: A](elem: A1): Boolean
Which explains why this code does ...
Advice
1
vote
4
replies
53
views
Attempting to add a module-info.java to spark.core.2.13 of Apache Spark: how to make scala accepting module delaration and gather its classes?
I'm willing to progress into the subject of introducing java modules into Apache Spark. They are needed for:
sorting and avoiding conflicts between dependencies.
allow Spark modules to be integrated ...
- 3,619
2
votes
2
answers
83
views
Read JSON in UpperCamelCase (Pascal case) using Jsonx.formatCaseClass from play-json-extensions
I am working on some code which reads from an external REST API that uses UpperCamelCase (a.k.a. Pascal case) for the keys in its JSON responses:
{"FirstKey": "a", "SecondKey&...
- 23.5k
Best practices
3
votes
4
replies
56
views
Best exception to throw in pattern matching in Scala
I wrote a function that takes an array of integers. If the function has at least two integers, it will flip the first two integers and return a new array. (This is a problem in Scala For The ...
- 1,177
1
vote
1
answer
99
views
scala.MatchError: TimestampNTZType (of class org.apache.spark.sql.types.TimestampNTZType$)
We are using Kafka to get avro data with the help of schema registry. After upgrading to Spark 3.3.2 from 2.4 the kafka consumer is failing with error -
scala.MatchError: TimestampNTZType (of class ...
- 897
4
votes
0
answers
118
views
sbt-jacoco counting synthetic methods
I saw one problem during project upgrade.
Before upgrade setup:
Java - 1.8
Scala - 2.12
sbt - 1.5.0
sbt-jacoco - 3.1.0
Overall test coverage in SonarQube: ~72%
After upgrade:
Java - 17
Scala - ...
- 35
2
votes
1
answer
59
views
Deserializing ScalaPB enum from int value and class
I am currently working on a Kryo SerDe for scalaPB (similar to twitter 'chill').
I struggle implementing the read interface
override def read(
kryo: com.esotericsoftware.kryo.Kryo,
input: ...
- 888
1
vote
0
answers
62
views
Scala 3 is it possible to statically create code using quotes?
My idea was to embed data in a class "statically" - i.e. completely independant of other components - and write it to file during compile.
Something like this:
def createForm()(using Quotes) ...
- 93
5
votes
0
answers
104
views
Scala compiler infinitely compiling
I am converting a Scala 2.13 project to Scala 3. Since I have made that change, the compiler has never finished compiling.
There were normal syntax errors before the compiler hangs, which I have now ...
- 929
1
vote
1
answer
100
views
Regular expression not catching case labels in file
I am working out of Scala For The Impatient (3rd Edition) in my spare time. (I am not taking a class.). I am processing a list of java files with Scala. Specifically, I am looking for case labels, ...
- 1,177
1
vote
2
answers
97
views
Using PPrint, can I exclude the named argument from Some ("value"), but not those of other case classes?
I am currently using the PPrint library to generate case classes to paste into my regression test Scala code, based on the values my existing code returns:
case class MyCaseClass(
byte: Option[...
- 23.5k
2
votes
1
answer
68
views
backslash count in raw string interpolation in scala
I am using Scala 3.3.7.
When I do
val str1 = raw"\\\" // compile error: unclosed string literal
It is considering the \" as a single character.
But when I do
val str1 = raw"\\\\&...
- 346
Tooling
0
votes
1
replies
50
views
Create a integrated test with ZIO and Quill
Hello I'm trying to create a integrated database test using testcontainer, my testcotainer is running with liquibase, my problem is about create quill connection, follow my code:
container config:
...
- 195
1
vote
1
answer
85
views
trying to read bigquery array colum and passing it as columns to fetch from spark dataframe
I have a bigquery table with array column named as "column_list "
ALTER TABLE `test-project.TEST_DATASET.TEST_TABLE`
ADD COLUMN column_list ARRAY<STRING>;
update `test-project....
- 4,759
0
votes
1
answer
95
views
col function error type mismatch: found string required Int
I am attempting to programmatically remove specific columns/fields from a dataframe (anything that starts with _), whether the field is in the root or in a struct, using the dropFields method.
For ...
- 8,903