All Questions
Tagged with sqlalchemy pandas
1,144 questions
0
votes
1
answer
63
views
sqlAlchemy to_sql with azure function returning collation error, but work without issues if executing just the python file
I have written a python method to form a pandas df and upsert it into sqlserver. This method works perfectly fine when I try to execute the python file alone. But it throws collation exception when I ...
- 1,365
0
votes
0
answers
38
views
Pandas - decimal.InvalidOperation: [<class 'decimal.ConversionSyntax'>] during read_sql
I am running an extraction of a table in Python using Pandas. I am getting this error:
decimal.InvalidOperation: [<class 'decimal.ConversionSyntax'>]
The code:
dataframes[table] = pd.read_sql(...
- 9
0
votes
0
answers
42
views
Terribly slow loading Pandas Data Frame with 150 columns & 5 millions rows into SQL Server
I am trying to load data from Pandas dataframe with 150 columns & 5 million rows. On my machine or prod serverless platform it is taking 4 to 5 hours to load into sql server table.
Table has no ...
- 1
0
votes
1
answer
43
views
Write to different schema in postgresql using pandas
hi i have a python code where i am reading some data from PostgreSQL table(schema of the tables in a db) and doing some transformations and then writing some data back to PostgreSQL table but in ...
- 43
1
vote
0
answers
82
views
Pandas DataFrame.to_sql randomly and silently failes without error message
I am trying to write several pandas dataframes (of the same format) to a postgres db, however some of them randomly are not written to the db. In that case to_sql silently fails and returns -1 (...
- 91
2
votes
3
answers
72
views
How do I store Decimal() values coming from SQL in CSV file with Python?
I have a simple chatbot that generates SQL queries and uses them on a database. I want to store the output in a .csv file and then download that file.
This is usually possible when I take my output ...
- 23
0
votes
1
answer
144
views
sqlalchemy - This result object does not return rows despite using SET NOCOUNT ON
I have a query that returns data in Miscrosoft SQL server management studio, but I'm getting the following error when trying to read in a query using pandas:
sqlalchemy.exc.ResourceClosedError: This ...
- 3,327
0
votes
2
answers
76
views
Reading a table from SQL Server into a DataFrame using SQLAlchemy
I have a table named "products" on SQL Server. I would like to read the table into a DataFrame in Python using SQLAlchemy.
The pandas.read_sql function has a "sql" parameter that ...
- 135
0
votes
2
answers
136
views
Problems to load large amount of data from SQL Server database
My current problem is with loading large amount of data from a table with about 5.000.000 rows from a SQL Server database.
The setup (which I can't influence) is:
0 GPU
4000 CPU
15.0 Gi memory
My ...
- 13
0
votes
0
answers
60
views
SQLAlchemy Teradata - Unable to use the df.to_sql functionality
I am attempting to use SQLAlchemy to load data from a pandas dataframe into a Teradata database table using the pandas.to_sql method.
import sqlalchemy as sa
username = "my_username"
...
- 439
0
votes
1
answer
73
views
Import XLS data to sqlite database using Python script
I'm a beginner in python3 and sqlite3 coding; I'm working to code a small program to manage my production process: I need to import database from my old xlsx file to the sqlite3 database to speed the ...
0
votes
0
answers
50
views
Database connection fails unless I manually connect to database first
I'm developing an application in Python that will interact with a MySQL database. I'm working with a local copy of the database, using MySQL Server 8.0.40, and I'm running into an issue where the ...
- 87
1
vote
0
answers
73
views
Use attribute name as column name for sqlalchemy in pandas read_sql()
I am using pd.read_sql() together with SQL alchemy. However, it is not using the attribute name of the mapping for the columns in the pandas dataframe, but the original SQL column name:
class DBtable(...
- 344
2
votes
1
answer
101
views
Incorrect syntax near 'CAST' when using pandas.to_sql
I'm trying to write some code to update an sql table from the values in a pandas dataframe. The code I'm using to do this is:
df.to_sql(name='streamlit_test', con=engine, schema='dbo', if_exists='...
- 3,327
2
votes
1
answer
1k
views
Python using oracledb to connect to Oracle database with Pandas DataFrame Error: "pandas only supports SQLAlchemy connectable (engine/connection)"
I am pretty new to Python and even newer to Pandas and am hoping for some guidance
My company has an on-prem DEV Oracle database that I am trying to connect to using Python & Pandas. After some ...
- 301