Questions tagged [csv]
The csv tag has no summary.
34 questions
0
votes
4
answers
494
views
What is a suitable format for writing large amounts of data within few milliseconds
I have a c++ code that needs to store some data whenever an event is triggered. The data contains about 3000 floating point values. So each of these values needs to be written in a file when the event ...
4
votes
1
answer
340
views
When do regression models outperform naive methods?
Case1:
I have the following task to do: Training by the consecutive 3 days to predict the each 4th day. Each day data represents one CSV file which has dimension 24x25. Every datapoints of each CSV ...
4
votes
5
answers
3k
views
The best way to handle exceptions?
I have the following method, which needs to return a List, but exceptions might occur along the way, and I don't want to handle them in my application, mainly because I don't even know how to handle ...
2
votes
8
answers
1k
views
Fail fast is brittle
I am creating a CSV consumer (with Java). There is one field / column that should contain one of the values "Rename" or "Move".
I implemented this by allowing mixed case of letters,...
6
votes
1
answer
365
views
What to do when test data takes up most of repo size?
TL;DR: Given a repo with a lot of big fixture tests, when should I look for a solution where the golden files are not part of the repo anymore? Where should they be stored?
The setup
Testing a ...
-1
votes
1
answer
193
views
Loading in and Autocompleting a CSV File React
I'm currently working on a React Application with Firebase that involves a ton of data manipulation and reading. One specific feature is the ability to autofill a form based on a very large CSV file (...
1
vote
4
answers
905
views
System Design: Efficient way to import hundreds of different CSV formats
OVERVIEW
We have multiple data providers who provide us information from a large set of data sources. These end data sources provide data in CSV format.
Since, the CSV formats are not consistent ...
2
votes
3
answers
2k
views
Back-end solution for pulling from CSV files
I'm building a data visualization that displays COVID information for the United States, at the city, state, and county level.
The ultimate source of truth are three CSVs published by the New York ...
-5
votes
1
answer
68
views
How is data stored prior to downloading?
I go to several sites that offer their data in XML, JSON, CSV formats. As shown here,
https://www.space.org/basicspacedata/query/class/boxscore/format/csv, I choose the format. Question is how is ...
0
votes
1
answer
106
views
A program design question: Good idea using HDFS in c for reading large data?
I have mainly three groups of CSV files (each file is divided into several small files): First group of CSV files have 600+ GB in total (MAYBE 200+ GB if in int, cause CSV calculates by char right?), ...
-2
votes
1
answer
334
views
Store line numbers of CSV in the database
In our environment, we often have to import CSV files into a database, where each line represents a record. Though there is no actual business use case for storing the original line numbers inside the ...
0
votes
1
answer
161
views
Generating CSV export
I created a CSV export that works like the code below. There is a LinkedHashMap where the keys are the column title and values are functions where certain properties are read.
By reording the lines ...
0
votes
2
answers
1k
views
Passing csv files between regions
I have a desktop app that generates numerous csv files. The application always runs on PCs that have been configured for the English (United Kingdom) region/format, but it's feasible that files might ...
50
votes
4
answers
46k
views
Should UTF-8 CSV files contain a BOM (byte order mark)?
Our line-of-business software allows the user to save certain data as CSV. Since there are a lot of different formats (all called "CSV") in use in the wild, we are tying to decide what the &...
1
vote
1
answer
159
views
Storing survey data from csv to xml
I've been tasked with taking a sample of survey results that are stored in CSV and to convert them to XML.
I'm still fairly new to XML, but have come up with a couple of layouts I could use and was ...