Create different arrays from csv file using python

Question

I want to create 6 arrays (one for each column) from my csv which looks like below:

The first column is the point name(Pt_name) and it contains strings. All the other are containing float numbers (Hz_angle, Vz_angle, Slope_Dist, Reference_Ht, Instrument_Height)

The purpose is to be able to perform mathematical operations between the elements of the float columns.

Any ideas?

S2,0.000000,98.799682,12.056200,1.700000,1.545000
1,0.000052,98.799806,12.056800,1.700000,1.545000
2,78.734236,99.822405,17.919000,0.000000,1.545000
3,78.861726,108.352791,17.213700,0.000000,1.545000
4,28.505234,91.249749,6.779200,0.000000,1.545000
5,34.011213,110.976488,6.840100,0.000000,1.545000
6,27.427827,106.134477,6.387900,0.000000,1.545000
48,0.926245,98.540506,25.413900,0.000000,1.545000
49,389.808941,99.812394,25.351000,0.000000,1.545000
S1,122.545107,99.563594,12.056300,1.700000,1.545000
50,200.000125,99.563463,12.058800,1.700000,1.545000
51,60.723043,95.842462,8.607300,0.000000,1.545000

before applying the below answers. I have tried many thinks but with no luck. As i realised my approach was totally wrong. en example of tries: #import csv #import ast #with open("Τελικό_(Final).txt", "r") as f: #reader = csv.reader(f, delimiter=',') #for row in reader: #print [ast.literal_eval(x.strip()) for x in row] #import csv #with open('Τελικό_(Final).txt','r') as f: # reader = csv.DictReader(f, delimiter=',') # rows = list(reader) # print [ast.literal_eval(x.strip()) for x in rows] and many more. — Mou Xaxa
– Mou Xaxa, Commented Jun 28, 2013 at 11:31

Glorfindel · Accepted Answer · 2023-01-06 21:02:29Z

2

If you want to do math with arrays, you should look into numpy. To load your data, you can do:

In [1]: import numpy as np

In [2]: numbers = ['Hz_angle', 'Vz_angle', 'Slope_Dist', 'Reference_Ht', 'Instrument_Height']

In [3]: dt = np.dtype([('Pt_name', np.bytes_, 5)] + [(name, np.float32) for name in numbers])

In [4]: data = np.loadtxt('/tmp/csv', delimiter=',', dtype=dt)

In [5]: data['Vz_angle']
Out[5]: 
array([  98.79968262,   98.79980469,   99.82240295,  108.35279083,
         91.24974823,  110.97648621,  106.13447571,   98.54050446,
         99.81239319,   99.563591  ,   99.5634613 ,   95.84246063], dtype=float32)

This reads the file into an array of records of type dt.

edited Jan 6, 2023 at 21:02

Glorfindel

22.8k13 gold badges97 silver badges124 bronze badges

answered Jun 28, 2013 at 9:50

Lev Levitsky

66.4k23 gold badges155 silver badges184 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

dwxw Over a year ago

Yes, Numpy's great for this sort of thing, use it all the time.

Mou Xaxa Over a year ago

Thank you very much for the quick reply. I am not familiar with numpy so as soon as i install it and test it, i will reply..

Mou Xaxa Over a year ago

Ok i tried it. When i do print Pt_name, Hz_angle, Vz_angle, Slope_Dist, Reference_Ht, Instrument_Height, it returns only the last line of data 51 60.723043 95.842462 8.607300 0.0 1.545

Lev Levitsky Over a year ago

@MouXaxa Apparently, you did something different from what I showed in the answer. In the answers, variables Pt_name, Hz_angle, etc. are not defined at all. On the other hand, data['Pt_name'] returns an array of all names, etc.

Mou Xaxa Over a year ago

It took me a while but finaly I figure it out!! It cannot find the file given in[4]: because the path contains greek letters.. I placed the file in c:\ and changed all the non english letters with english. Now it works like a charm!. Appreciate your help and tips!!!

|

martyn · Accepted Answer · 2013-06-28 12:13:55Z

Didn't test it, but something like this should work:

import csv

Pt_names = []
Hz_angles = []
Vz_angles = []
Slope_Dists = []
Reference_Hts = []
Instrument_Heights = []
with open(csv_file, 'r') as fh:
    reader = csv.reader(fh)
    for row in reader:
        Pt_name = row[0]
        # list comprehension for float conversion
        Hz_angle, Vz_angle, Slope_Dist, Reference_Ht, Instrument_Height = [float(value) for value in row[1:]]
        Pt_names.append(Pt_name)
        Hz_angles.append(Hz_angle)
        Vz_angles.append(Vz_angle)
        Slope_Dists.append(Slope_Dist)
        Reference_Hts.append(Reference_Ht)
        Instrument_Heights.append(Instrument_Height)

I have just tried it it works great but it returns NameError: name 'Pt_name' is not defined. I do not want to convert pt_names to float.

Collectives™ on Stack Overflow

Create different arrays from csv file using python

2 Answers 2

6 Comments

1 Comment

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

6 Comments

1 Comment

Related