User WestCoastProjects - Stack Overflow

109 votes

Accepted

Is it possible to get the current spark context settings in PySpark?

answered Jun 1, 2015 at 2:43

70 votes

Accepted

Number of partitions in RDD and performance in Spark

answered Mar 4, 2016 at 17:23

67 votes

How are stages split into tasks in Spark?

answered May 30, 2016 at 15:27

58 votes

Accepted

How to set Master address for Spark examples from command line

answered Jun 30, 2014 at 0:55

52 votes

Accepted

How to restore default layout for intellij Run/Debug window/tabs?

answered Feb 7, 2017 at 20:07

47 votes

Pythonic way to combine for-loop and if-statement

answered Mar 4, 2018 at 21:44

46 votes

Accepted

Show partitions on a pyspark RDD

answered Mar 15, 2015 at 2:06

41 votes

Accepted

Recursively fetch file contents from subdirectories using sc.textFile

answered Mar 3, 2015 at 23:55

41 votes

How to match a string, but case-insensitively?

answered Mar 4, 2014 at 0:06

30 votes

Accepted

How to install sbt on ubuntu/debian with apt-get

answered Feb 20, 2016 at 23:43

25 votes

Neo4j sharding aspect

answered Dec 28, 2017 at 18:43

23 votes

How to run a single test in scalatest from maven

answered Jul 20, 2014 at 17:57

22 votes

Accepted

How to create a Spark Dataset from an RDD

answered May 29, 2016 at 19:05

22 votes

How to check if collection contains any element from other collection in Scala?

answered Jun 17, 2016 at 19:05

21 votes

Accepted

Test accuracy is greater than train accuracy what to do?

answered Jul 22, 2018 at 19:08

21 votes

Accepted

Why is "abstract override" required not "override" alone in subtrait?

answered May 14, 2014 at 5:40

20 votes

Accepted

Addition of two RDD[mllib.linalg.Vector]'s

answered Jan 31, 2015 at 1:35

20 votes

Accepted

Json serialization error using matplotlib mpld3 with LinkedBrush

answered Nov 19, 2017 at 21:12

20 votes

Accepted

How to run a Scala script within IntelliJ IDEA?

answered Nov 20, 2013 at 18:19

20 votes

Accepted

Hive command to execute NOT IN clause

answered Jan 2, 2014 at 14:49

19 votes

Hadoop DistCp using wildcards?

answered Jan 5, 2014 at 20:55

19 votes

Accepted

How to convert a Scala Array to ArrayBuffer?

answered Feb 4, 2018 at 20:02

19 votes

Accepted

How to sort an RDD in Scala Spark?

answered May 24, 2014 at 7:23

18 votes

Why does Spark fail with java.lang.OutOfMemoryError: GC overhead limit exceeded?

answered Dec 13, 2014 at 18:55

18 votes

What do columns ‘rawPrediction’ and ‘probability’ of DataFrame mean in Spark MLlib？

answered Jun 19, 2016 at 17:21

17 votes

Is there a 'foreach' function in Python 3?

answered Dec 19, 2020 at 10:14

15 votes

Accepted

UnsatisfiedLinkError: no snappyjava in java.library.path when running Spark MLLib Unit test within Intellij

answered May 4, 2015 at 21:07

15 votes

Accepted

Spark : check your cluster UI to ensure that workers are registered

answered Feb 27, 2016 at 5:54

14 votes

Multiple table join in hive

answered Mar 13, 2015 at 20:49

13 votes

Spark : how to run spark file from spark shell

answered Dec 31, 2014 at 16:43

Collectives™ on Stack Overflow

782 Answers

Is it possible to get the current spark context settings in PySpark?

Number of partitions in RDD and performance in Spark

How are stages split into tasks in Spark?

How to set Master address for Spark examples from command line

How to restore default layout for intellij Run/Debug window/tabs?

Pythonic way to combine for-loop and if-statement

Show partitions on a pyspark RDD

Recursively fetch file contents from subdirectories using sc.textFile

How to match a string, but case-insensitively?

How to install sbt on ubuntu/debian with apt-get

Neo4j sharding aspect

How to run a single test in scalatest from maven

How to create a Spark Dataset from an RDD

How to check if collection contains any element from other collection in Scala?

Test accuracy is greater than train accuracy what to do?

Why is "abstract override" required not "override" alone in subtrait?

Addition of two RDD[mllib.linalg.Vector]'s

Json serialization error using matplotlib mpld3 with LinkedBrush

How to run a Scala script within IntelliJ IDEA?

Hive command to execute NOT IN clause

Hadoop DistCp using wildcards?

How to convert a Scala Array to ArrayBuffer?

How to sort an RDD in Scala Spark?

Why does Spark fail with java.lang.OutOfMemoryError: GC overhead limit exceeded?

What do columns ‘rawPrediction’ and ‘probability’ of DataFrame mean in Spark MLlib？

Is there a 'foreach' function in Python 3?

UnsatisfiedLinkError: no snappyjava in java.library.path when running Spark MLLib Unit test within Intellij

Spark : check your cluster UI to ensure that workers are registered

Multiple table join in hive

Spark : how to run spark file from spark shell