Importing data from multiple csv files in D3

Question

I am brand new to D3 and just started working on an a project. My question is this. I want to import data from 2 csv files in D3 to use them for graph comparisons. The problems I am facing are:

1.How do I import data from multiple csv files.
2.Can I use one array for each csv or does D3 use only one global data array?
3.Is there a way to choose a certain column from the csv files to import?

Here is an example, I want to import the "oldVer" from each of the files in separate arrays and then use the 2 arrays to work with. Is that posible in D3 and how?

csv 1
time,oldVer,newVer,oldT,newT
1,180930,190394,24,59
2,198039,159094,26,45
3,152581,194032,22,61

csv 2
time,oldVer,newVer,oldT,newT
1,184950,180435,27,26
2,120590,129409,13,13
3,165222,182133,60,54

Again sorry for the dumb question but I have found little feedback on this matter. Any help will be appreciated.

Abang F. · Accepted Answer · 2018-06-30 08:48:17Z

38

In d3 version 5, you can use Promise.all to load multiple csv files. Example:

Promise.all([
    d3.csv("file1.csv"),
    d3.csv("file2.csv"),
]).then(function(files) {
    // files[0] will contain file1.csv
    // files[1] will contain file2.csv
}).catch(function(err) {
    // handle error here
})

More info about loading csv in d3 v5

More info about Promise.all()

answered Jun 30, 2018 at 8:48

Abang F.

9168 silver badges13 bronze badges

1

Thanks @Abang F, the simplest and clearest demonstration I've seen :)
– Markus
Commented Jul 11, 2018 at 7:57
Can I use this somehow with a dynamic number of files?
– timbmg
Commented Oct 23, 2018 at 10:22
1

@blckbird Promise.all(fileList.map(d3.csv)).then(...)
– BallpointBen
Commented Jun 12, 2020 at 4:33

Add a comment |

Lars Kotthoff · Accepted Answer · 2014-02-18 01:27:25Z

17

You simply call d3.csv several times:

d3.csv("csv1.csv", function(error1, data1) {
  d3.csv("csv2.csv", function(error2, data2) {
    // do something with the data
  });
});

As for your third question, no, D3 will parse everything. There's nothing forcing you to use all the data though, so if you're interested in only one column, just use the data from that.

answered Feb 18, 2014 at 1:27

Lars Kotthoff

109k16 gold badges210 silver badges208 bronze badges

1

Hi Lars, since the objective is to compare the data and thus have both available at the same time, there would be a need to control the flow of execution of these nested asynchronous calls, no? If we do something inside the inner function, there is no guarantee that data1 will be ready. Am I right? I was reading this. It is talking about node.js but perhaps it would apply.
– FernOfTheAndes
Commented Feb 18, 2014 at 2:39
Right on spot, I will try it out now. I've read about asynchronous calls before, but cannot say much, I just have to try it out. Thank you for the quick responce guys.
– John M.
Commented Feb 18, 2014 at 11:14
IF I am right here, in the link I referenced, I believe you want 7.2.2 Control flow pattern #2, where "...we just want to take a small set of operations, launch them all in parallel and then do something when all of them are complete."
– FernOfTheAndes
Commented Feb 18, 2014 at 12:11
1

@FernOfTheAndes No -- this is exactly what the nesting takes care of. I'm sending the request for the second CSV only when the first request is complete, so inside the second handler function both are guaranteed to be available.
– Lars Kotthoff
Commented Feb 18, 2014 at 13:53
I see now...the inner call will only run when the data returns from the outer call. Sorry for the confusion.
– FernOfTheAndes
Commented Feb 18, 2014 at 14:15

| Show 2 more comments

Stuart Hallows · Accepted Answer · 2016-09-14 06:45:13Z

15

You could use a d3 queue to load the files simultaneously. An example;

d3.queue()
.defer(d3.csv, "file1.csv")
.defer(d3.csv, "file2.csv")
.await(function(error, file1, file2) {
    if (error) {
        console.error('Oh dear, something went wrong: ' + error);
    }
    else {
        doStuff(file1, file2);
    }
});

edited Sep 14, 2016 at 6:45

answered Sep 14, 2016 at 6:40

Stuart Hallows

8,9735 gold badges47 silver badges57 bronze badges

1

does this work for json as well? e.g. ``` d3.queue()/* to load multiple datasets */ .defer(d3.json, "data1") .defer(d3.json, "data2") .await(function(error, data1, data2) { if (error) { console.log('Something went wrong: ' + error); } else { //stuff for graph } });```
– thesayhey
Commented Feb 23, 2017 at 22:42
Great way of doing that!
– user2095363
Commented Aug 2, 2017 at 6:46

Add a comment |

FrankS101 · Accepted Answer · 2018-03-19 20:40:05Z

1

To answer your part 3,

Is there a way to choose a certain column from the csv files to import?

No, you cannot load in part of a CSV. You can, however, load in the entire CSV file and selectively use one column from it. You can refer to data.newVer to utilize the newVer column data.

edited Mar 19, 2018 at 20:40

FrankS101

2,1556 gold badges27 silver badges42 bronze badges

answered Mar 19, 2018 at 19:05

travelsize

741 silver badge4 bronze badges

Add a comment |

Collectives™ on Stack Overflow

Importing data from multiple csv files in D3

4 Answers 4

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Linked

Related