How to import an array of arrays from CSV in Python

Question

so basically I'm new to Python and there's something I'm not being able to do. I'm importing data from a CSV and I need my data_2d to look like this:

data_2d = [ [30, 15, 0, 15, 0, 0, 0],
        [32, 10, 0,10, 3, 5, 0],
        [5, 9, 0, 25, 10, 8, 3],
        [22, 10, 0  ,17, 5, 6, 0],
        [7, 15, 0, 30, 3, 5, 0]]

Instead, with my current code, I get this:

[['30' '15' '0' '15' '0' '0' '0']
['32' '10' '0' '10' '3' '5' '0']
['5' '9' '0' '25' '10' '8' '3']
['22' '10' '0' '17' '5' '6' '0']
['7' '15' '0' '30' '3' '5' '0']]

My code is here:

data_2d = [ [30, 15, 0, 15, 0, 0, 0],
        [32, 10, 0,10, 3, 5, 0],
        [5, 9, 0, 25, 10, 8, 3],
        [22, 10, 0  ,17, 5, 6, 0],
        [7, 15, 0, 30, 3, 5, 0]]

data_2d = []

with open('file.csv', newline='\n') as f:
    reader = csv.reader(f, delimiter=';')
    for row in reader:
        data_2d.append(row)

data_array = np.array(data_2d)
data_array = np.delete(data_array, (0), axis=0)
data_array = np.delete(data_array, (0), axis=1)

print("data_array")
print(data_array)

The CSV file currently looks like this:

Time_Activity;SITTING;STANDING;LAYING;WALKING;WALKING_DOWNSTAIRS;WALKING_UPSTAIRS;RUNNING
8;30;15;0;15;0;0;0
9;32;10;0;10;3;5;0
10;5;9;0;25;10;8;3
11;22;10;0;17;5;6;0
12;7;15;0;30;3;5;0

Naveen · Accepted Answer · 2019-02-05 15:26:54Z

3

The csv file is read as a string. You can convert the row from string to int while appending. This can be done using map function.

Code:

data_2d.append(list(map(int,row)))

answered Feb 5, 2019 at 15:26

Naveen

1,2108 silver badges22 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

jpp · Accepted Answer · 2019-02-05 15:46:14Z

For an immediate fix, use the astype method to convert your array of strings to int:

data_array = data_array.astype(int)

In addition, np.delete is inefficient and not recommended; use slicing instead where possible. Here are a couple of ways you can avoid Python-level loops altogether:-

`numpy`

Using NumPy, you can utilize np.genfromtxt:

arr = np.genfromtxt('file.csv', delimiter=';', skip_header=1)[:, 1:]

`pandas`

Alternatively, via Pandas you can use pd.read_csv:

arr = pd.read_csv('file.csv', sep=';').iloc[:, 1:].values

print(arr)

# array([[30, 15,  0, 15,  0,  0,  0],
#        [32, 10,  0, 10,  3,  5,  0],
#        [ 5,  9,  0, 25, 10,  8,  3],
#        [22, 10,  0, 17,  5,  6,  0],
#        [ 7, 15,  0, 30,  3,  5,  0]], dtype=int64)

foobarbaz · Accepted Answer · 2019-02-05 15:37:34Z

You are on the right track. 2 things can help achieve your goal and simplify the code a bit.

Skip the header so you don't worry about deleting it later.
Cast the strings to int before you append.

Example code that works on your input:

  data_2d = []
  with open('file.csv', newline='\n') as f:
    reader = csv.reader(f, delimiter=';')
    next(reader) # this will skip that header
    for row in reader:
      data_2d.append([int(x) for x in row]) #this just means take each string and make it an int before appending.

Result:

[[8, 30, 15, 0, 15, 0, 0, 0], 
[9, 32, 10, 0, 10, 3, 5, 0], 
[10, 5, 9, 0, 25, 10, 8, 3], 
[11, 22, 10, 0, 17, 5, 6, 0], 
[12, 7, 15, 0, 30, 3, 5, 0]]

Some helpful links that explain both additions: https://evanhahn.com/python-skip-header-csv-reader/ https://www.pythonforbeginners.com/basics/list-comprehensions-in-python

Collectives™ on Stack Overflow

How to import an array of arrays from CSV in Python

3 Answers 3

Comments

`numpy`

`pandas`

Comments

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

numpy

pandas

Comments

Comments

Linked

Related

`numpy`

`pandas`