Writing Numpy Array with NaNs into CSV in python

Question

I am trying to write 2D numpy array into csv file using np.savetxt.

import numpy as np

data = np.array([[0,np.nan,2,3],[4,5,np.nan,7],[8,9,10,np.nan]])

np.savetxt("file.csv", data, delimiter=",", fmt='%.2f')

Which creates a file file.csv with the following content.

0.00,nan,2.00,3.00
4.00,5.00,nan,7.00
8.00,9.00,10.00,nan

As you can see the file contains nan instead of blanks. I know nan is not a string.

Expected Output:

0.00,,2.00,3.00
4.00,5.00,,7.00
8.00,9.00,10.00,

Using pandas I can achieve this like below.

import pandas as pd

df = pd.DataFrame(data)
df.to_csv("file1.csv", index=False)

But I'm refraining from using pandas for now. So is it possible to achieve this using numpy?

All savetxt is doing is a formatted write of each row of your array. '%.2f,%.2f,...'%tuple(row). Just basic python % formatting. — hpaulj
– hpaulj, Commented Mar 21, 2019 at 5:02

DYZ · Accepted Answer · 2019-03-21 04:48:05Z

6

The most straightforward solution:

data1 = data.astype(str)
data1[data1=='nan'] = ''
np.savetxt("file.csv", data1, delimiter=",", fmt="%s")

answered Mar 21, 2019 at 4:43

DYZ

57.3k10 gold badges73 silver badges101 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

I hope it's not gonna affect the file size converting them into string and storing them?

The file is a text file. Numpy implicitly converts all numbers to strings before writing them, anyway.

With %s you loose control over column width and decimal precision, the kinds of things that %.2f formats provide.

Tomerikoo · Accepted Answer · 2021-12-30 18:32:49Z

0

You can solve this using pandas with na_rep parameter. For example:

df.to_csv(r'path\df.csv', sep=',', na_rep=np.NaN)

Tomerikoo

19.6k16 gold badges57 silver badges68 bronze badges

answered Dec 30, 2021 at 18:10

MClaver89

614 bronze badges