How to flatten an ArrayBuffer in Scala

Question

Here is the sample piece of code. I am using an array of RDDS to store objects. I would like to use flatMap on this array of RDDS and store that into a dataframe.

var ArrayofRDDs: ArrayBuffer[T] = null
for(i <- 1 to 10) {
  val data = <A function that returns an object of type T>
  ArrayofRDDs :+ data
}

Can someone help with how to use flatMap on the Array of RDDs?

zero323 · Accepted Answer · 2016-03-30 05:50:00Z

2

how to use flatMap on the Array of RDDs

Simply don't. RDDs are not GenTraversableOnce and cannot be converted to one without using some variant of collect. If you want to combine multiple RDDs just use SparkContext.union.

answered Mar 30, 2016 at 5:50

zero323

331k108 gold badges982 silver badges958 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How to flatten an ArrayBuffer in Scala

1 Answer 1

Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Related