python requests response.text logging "Encoding detection: utf_8 is most likely the one."

Question

(my solution below)

I have a python script that uses a REST API to get XML files from a tool. This is the heart of it:

def get_output(self, url, output_filename):
    xml_response = self.client.get_request(url)
    if not xml_response:
        self.logger.error(f"no xml_response:{output_filename}")
        return False
    else:
        self.logger.info(f"{output_filename}")
        output_file = open(output_filename, 'w', encoding='utf-8')
        output_file.write(xml_response.text)
        output_file.close()
        return True

The write call causes a DEBUG log entry:

Encoding detection: utf_8 is most likely the one.

Is there a way to explicitly tell the write to use utf8 in order to not get that log entry?

Thanks!

Edit 1: self.client.get_request(url) is essentially:

import requests
self.headers = {'Accept': 'application/rdf+xml'}
self.session = requests.Session()
response = self.session.get(url, allow_redirects=True, headers=self.headers)

This is a sanitized log entry:

2025-09-08T22:00:52+0000;INFO;XXX:./__auto_temp/2025-09-08_2200(+0000)___XXX.xml
2025-09-08T22:01:36+0000;DEBUG;https://XXX "GET /XXX HTTP/1.1" 200 None
2025-09-08T22:01:36+0000;INFO;./__auto_temp/2025-09-08_2200(+0000)___XXX.xml
2025-09-08T22:01:36+0000;INFO;get_output 1
2025-09-08T22:01:36+0000;INFO;get_output 2
2025-09-08T22:01:36+0000;DEBUG;Encoding detection: utf_8 is most likely the one.
2025-09-08T22:01:36+0000;INFO;get_output 3
2025-09-08T22:01:36+0000;INFO;get_output 4

Unless log entries are coming in out-of-order, my experiment that sprinkled logger calls among the open/write/close show it's the write causing the entry.

self.logger.info(f"{output_filename}")
self.logger.info('get_output 1')
output_file = open(output_filename, 'w', encoding='utf-8')
self.logger.info('get_output 2')
output_file.write(xml_response.text)
self.logger.info('get_output 3')
output_file.close()
self.logger.info('get_output 4')
return True

Edit 2: My solution based on deceze's answer was to add xml_response.encoding = 'utf-8' before the write; the log message no longer appears. I changed the title of the post to reflect the actual problem. I did not realize text was a property with some code behind it, I thought it was a simple get to the buffer.

Is there any more context here? Any file/line number where this message is being generated exactly? — deceze
– deceze ♦, Commented Sep 9 at 9:02
BTW, f"{output_filename}" is pretty redundant. Just self.logger.info(output_filename) will do the same thing. — deceze
– deceze ♦, Commented Sep 9 at 9:02
Please edit your question and add more debugging details. In its current shape, we can not reproduce the issue stackoverflow.com/help/minimal-reproducible-example — Ajeet Verma
– Ajeet Verma, Commented Sep 9 at 9:10
What you posted is not an error and not even the full log entry. What's going on and why do you assume it's relevant? Googling for the exact message returns results like [charset_normalizer] DEBUG: Encoding detection: utf_8 is most likely the one. that come after HTTP requests. Perhaps the API you call didn't specify a Content-Type, so whatever client is, tried to detect the encoding ? — Panagiotis Kanavos
– Panagiotis Kanavos, Commented Sep 9 at 9:35
What are client and get_request ? Does the package they came from use charset-normalizer And what are the full log entries ? — Panagiotis Kanavos
– Panagiotis Kanavos, Commented Sep 9 at 9:39

deceze · Accepted Answer · 2025-09-09 10:29:56Z

That log entry is produced by accessing response.text, not by the write operation:

When you make a request, Requests makes educated guesses about the encoding of the response based on the HTTP headers. The text encoding guessed by Requests is used when you access r.text. You can find out what encoding Requests is using, and change it, using the r.encoding property.

https://requests.readthedocs.io/en/latest/user/quickstart/#response-content

If you want to read the response as text, that is inevitable. Dial down your logging level to exclude debugging messages, if you don't like it.

The alternative would be to forego decoding the response as text, and save it directly in its originally transferred binary form:

                            👇
with open(output_filename, 'wb') as output_file:
    output_file.write(xml_response.content)
                                      👆

thank you, i will test the solution tonight since the tool restricts REST access during biz hours.
i put my solution as an edit to the post. many thanks @deceze, you were a great help.
I'd've just dialled down the logging level, or ignored it. You'll get a lot of these kinds of "too much information" logs with DEBUG enabled; that's what it's for. But you do you…

Collectives™ on Stack Overflow

python requests response.text logging "Encoding detection: utf_8 is most likely the one."

1 Answer 1

3 Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Related