2

I am trying to build a d3.js heatmap from a CSV dataset. Upon import using d3.csv, the data looks like this:

[Object, Object, Object, Object, Object, Object, Object, Object, Object, Object, columns: Array(11)]

where each object is as follows:

Object {0: "0.934999642", 1: "0.1031451091", 2: "0.077435557", 3: 
"0.129644215", 4: "0.666245944", 5: "0.133087829", 6: "0.113218775", 7: 
"0.120796943", 8: "0.257501418", 9: "0.840916491", "": "0"}

I have been trying to use the data as is, but for loops are too slow (the real dataset has 1300 values), and attempting a nesting structure doesn't work because d3's data() function can only take an array.

Like so:

var row =  svg.selectAll(".rows")
                     .data(data)
                     .enter()
                     .append("rect")
                     [etc]

d3.selectAll(".blocks")//nested data
                     .data(function(d) {//d has no length property, so this line fails
                        return d;
                     })
                     .enter()
                     .append("rect")
                    [etc] 

How can I convert the array of objects into an array of arrays? Alternatively, is there a way to use the array of objects successfully? Also, is there any reason why it isn't coming directly from the CSV in an array of arrays rather than this weird object array? The data was curated in and exported from Python's Pandas, if that makes a difference, and we can make adjustments there.

Thank you.

6
  • 3
    This is clearly an XY problem. Instead of asking what you are asking right now, please explain exatcly what you want to draw with that data. Besides that, the answer to "is there any reason why it isn't coming directly from the CSV in an array of arrays rather than this weird object array?" is yes, that's precisely what d3.csv does. Commented Jun 11, 2017 at 19:15
  • I am trying to build a d3.js heatmap from a CSV dataset. How do I do this given the dataset I am working with?
    – K Hutch
    Commented Jun 11, 2017 at 19:17
  • Thank you for clarifying the CSV import issue. Because of how the dataset was curated, I have been trying to figure out if there was a problem on that end that was resulting in an unexpected d3.csv import structure. It is good to know that this data structure is what is expected.
    – K Hutch
    Commented Jun 11, 2017 at 19:20
  • I added an answer, but I encourage you to look around for similar questions before posting one. This will probably be marked as a duplicate. Commented Jun 11, 2017 at 21:41
  • 1
    Well, I see you have 4 answers so far, and they are all valid, because they answer your actual question. However, since you told that you want to create a heatmap, I predict that none of them will help you. To avoid the XY problem, post a question clearly showing your data structure, your enter selections and explaining what you want to achieve. Preferably, a new question. Commented Jun 12, 2017 at 0:20

5 Answers 5

5

This is straightforward enough with a functional style using Array.prototype.map:

let myArray = Object
    .keys(myObj)
    .map(key => myObj[key]);

Though you may want to guarantee that the keys appear in order and exclude non-numeric keys.

let myArray = Object
    .keys(myObj)
    .sort((a,b) => +a - +b)
    .filter(key => !isNaN(+key))
    .map(key => myObj[key]);

You can in-turn use this with map to apply it to the parent array

let newArray = bigArray
    .map(el => Object
        .keys(el)
        /* ... */
        .map(key => el[key]));
0

myObj = {prop1:[object, object,...], prop2: ''} ;

to get prop1 as array, use this:

var result = myObj.prop1;

or result = myObj.prop1.map(function (item) { /item - your each object, you can get any property, like: return {"name": item. name, "value" : item[3];} ;/ return item;}) ;

result = [{name: Alex, value: Cash},..] ;

1
  • Welcome to Stack Overflow. Please take a look at the formatting help to properly format your answers in the future, as well as the help page on answering to get a better idea of how to write a good answer. Your answer in its current state is very low quality and is bound to get downvoted into oblivion. Cheers. Commented Jun 11, 2017 at 22:17
0

An Array of Arrays. Using a While loop with object iteration and array assignment. Technically two loops.Practically should be counted as one because it is a continuation of a nested count.

var objs = [{
  0: "0.934999642",
  1: "0.1031451091",
  2: "0.077435557",
  3: "0.129644215",
  4: "0.666245944",
  5: "0.133087829",
  6: "0.113218775",
  7: "0.120796943",
  8: "0.257501418",
  9: "0.840916491",
  "": "0"
}, {
  0: "0.934999642",
  1: "0.1031451091",
  2: "0.077435557",
  3: "0.129644215",
  4: "0.666245944",
  5: "0.133087829",
  6: "0.113218775",
  7: "0.120796943",
  8: "0.257501418",
  9: "0.840916491",
  "": "0"
}]

var i = objs.length,
  arr = Array.from(new Array(i), x => []);
while (i--) {
  for (let val in objs[i]) {
    var values = objs[i][val];
    arr[i].push([values]);
  }
}
console.log(arr);

5
  • Your solution seems awful complicated, and includes a for..in loop, which is a statement that should be avoided altogether. Use Object.keys instead. Object.keys(obj).forEach(function(key){ ... }) is cleaner and more predictable. Commented Jun 11, 2017 at 22:38
  • a while loop is complicated? What reason is there to avoid the "for in loop" in this instance? Object.keys is good shorthand and similarly so for forEach, However, the question was one of performance, and you sacrifice that with the shorthand you have selected.
    – Rick
    Commented Jun 11, 2017 at 22:44
  • Sounds like Premature optimization to me. You should be optimizing where big gains can be made, and only after you have a functioning product. Apply the 80:20 rule and don't needlessly peck away at code that will only net you diminishing returns. Otherwise you end up with an unmaintainable mess. Plus, if you're really worried about squeezing out every possible ounce of performance, we have asm.js, which you can target with a compiler for another language, which will be far more optimized than anything a human can do. Commented Jun 11, 2017 at 22:57
  • You think it's "premature" to assume he is dealing with large data when he says he is dealing with mapping large amounts of data from "CSV" files. Maybe when he mentions "for loops are too slow". I don't know, maybe he needs something even slower... I don't know... Maybe a suboptimal approach and obfuscation are exactly what this questioner is really asking for. In which case you have him covered :-)
    – Rick
    Commented Jun 11, 2017 at 23:14
  • Unless you think minifiers and compilers that target asm.js are somehow "obfuscation" (which is a ludicrous viewpoint in itself), I have no idea where you got the idea that I'm obfuscating. Code should be clean and maintainable first. And when you're optimizing, you should aim for opportunities to save seconds, not microseconds. Again, 80:20 rule. 80% of your optimizations will affect 20% of the code; the rest is not worth the bother and you'll be beaten out by automated tools every time. If you really think your "optimizations" are effective, I'd like to see some benchmarks to prove it. Commented Jun 11, 2017 at 23:28
0

You already have 4 different answers showing you how to convert an object with several properties in an array. However, as I said in my comment, this seems to be an XY problem to me, since what you really want at the end is just creating a heatmap.

Of course you can use any of the provided answers to create your data array in your inner selection. However, there are different ways to achieve the same result. This answer is just my two cents regarding your objective, not regarding what you actually asked.

You are correct when you said that data() doesn't accept an object. In fact, data() only accepts three things:

  1. An array;
  2. A function;
  3. Nothing.

That being said, we have to somehow convert that huge object in several objects, since each individual object represents a data point.

Preparing your data array

Suppose you have the following CSV (an 8 x 8 matrix of values). In this CSV, the first row, the header, is just the column number. All the values (from the second row on) go from 0 to 100:

1, 2, 3, 4, 5, 6, 7, 8
99, 01, 01, 56, 65, 34, 87, 65
76, 99, 73, 23, 54, 62, 12, 15
54, 77, 66, 23, 98, 76, 11, 12
13, 24, 56, 76, 88, 75, 43, 76
12, 33, 44, 52, 45, 87, 65, 67
88, 76, 67, 87, 54, 34, 44, 45
12, 13, 24, 65, 45, 47, 76, 87
65, 43, 23, 42, 42, 34, 42, 17

With this CSV, you can create a data array in which each object contains the row number, the column number and the value. For instance, parsing the CSV to an array named rawData, we can populate the final data array, named data, like this:

rawData.forEach(function(d, i) {
    Object.keys(d).forEach(function(e) {
        data.push({
            row: i,
            column: e,
            value: d[e]
        })
    })
});

Thus, you only need to access d.row, d.column and d.value in your selection.

.attr("x", function(d) {
    return xScale(d.column)
})
.attr("y", function(d) {
    return xScale(d.row)
})
.attr("fill", function(d) {
    return color(d.value)
})

Here is a very simple demo:

var svg = d3.select("svg");
var rawData = d3.csvParse(d3.select("#csv").text())
var color = d3.scaleLinear().domain([0, 100]).range(["maroon", "lawngreen"]);
var xScale = d3.scaleBand().domain(d3.range(1, 9, 1)).range([20, 180]);
var yScale = d3.scaleBand().domain(d3.range(1, 9, 1)).range([20, 180]);

var data = [];

rawData.forEach(function(d, i) {
  Object.keys(d).forEach(function(e) {
    data.push({
      row: i,
      column: e,
      value: d[e]
    })
  })
});

var rects = svg.selectAll(null)
  .data(data)
  .enter()
  .append("rect")
  .attr("width", xScale.bandwidth() - 2)
  .attr("height", yScale.bandwidth() - 2)
  .attr("x", function(d) {
    return xScale(d.column)
  })
  .attr("y", function(d) {
    return xScale(d.row)
  })
  .attr("fill", function(d) {
    return color(d.value)
  })
pre {
  display: none;
}
<script src="https://d3js.org/d3.v4.min.js"></script>
<svg width="200" height="200"></svg>
<pre id="csv">1,2,3,4,5,6,7,8
99,01,01,56,65,34,87,65
76,99,73,23,54,62,12,15
54,77,66,23,98,76,11,12
13,24,56,76,88,75,43,76
12,33,44,52,45,87,65,67
88,76,67,87,54,34,44,45
12,13,24,65,45,47,76,87
65,43,23,42,42,34,42,17</pre>

Note that in this demo I'm using a <pre> element to store the data, since I cannot use d3.csv in a Stack snippet.

0

You could iterate over the Array and map each object to the Object values:

var result=array.map(Object.values)

But its the same speed as two loops.

5
  • You need to make it clearer that array refers to the data structure in OP's example. I almost downvoted your answer with a lecture on "why this answer is incorrect" as a result of the lack of information, because I thought you were trying to call Array.map with a function as its input and somehow magically get the wanted data. Unclear intent is a good way to fail a code review. Commented Jun 11, 2017 at 22:00
  • @braden best i admit that i left away the explanantion. However, thats still no reason to downvote. Commented Jun 12, 2017 at 14:21
  • I didn't downvote, but -- what's no reason to downvote? I almost downvoted this answer for being incorrect (and that is the only reason I will downvote an answer), not for ambiguity. But I am thorough, and didn't end up downvoting because I was eventually able to verify that your answer was correct. I make great effort to avoid erroneously downvoting an answer. Commented Jun 12, 2017 at 21:12
  • @braden best ive never downvoted an answer. If its unhelpful / wrong i comment and explain why i think its wrong, if its impossible to reach the answerer i usually edit myself / flag. Thats the helpfullwst for all... Commented Jun 13, 2017 at 17:59
  • 1
    Come to think of it, I don't think I've ever downvoted an answer, either. So I guess that makes me a hypocrite for singling you out. Fair enough. Commented Jun 13, 2017 at 18:51

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.