I am trying to reverse engineer a web app. So far, using the inspect tool on my browser, I have managed to log in the website using python and use multiple parts of the application.
Short example:
# Log in
session = requests.Session()
login_response = session.request(method='POST', url=LOGIN_URL, data=build_login_body())
session.cookies = login_response.cookies
# Call requests post method
session.request(method='POST', url=URL_1, data=build_keyword_update_body(**kwargs),
headers={'Content-type': 'application/json; charset=UTF-8'}
)
However there is one URL (URL_2
) for which if I only pass the content-type headers then I get a 'HTTP 400 Bad Request Error'. To work around that, I copied all the headers used in the inspect tool and made a request as follows:
session.request(
method='POST',
url=URL_2,
data={},
headers={
'accept': '*/*',
'cookie': ';'.join([f'{cookie.name}={cookie.value}' for cookie in session.cookies]),
'origin': origin_url,
'referer': referer_url,
'sec-ch-ua': 'Not A;Brand";v="99", "Chromium";v="100", "Google Chrome";v="100',
'sec-ch-ua-mobile': '?0',
'sec-ch-ua-platform': 'macOS',
'sec-fetch-dest': 'empty',
'sec-fetch-mode': 'cors',
'sec-fetch-site': 'same-origin',
'content-type': 'application/json; charset=UTF-8',
'accept-language': 'en-GB,en-US;q=0.9,en;q=0.8',
'accept-encoding': 'gzip, deflate, br',
'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.127 Safari/537.36'
}
The headers above give me a 401 Unauthorized error. I found out that if I remove the user-agent header I get a bad request, but when I add it I get the 401 Unauthorized error.
I tried adding the same user-agent in all requests' headers, including login, but it didn't help. I also tried passing an HTTPBasicAuth
or HTTPDigestAuth
object to the request parameters as well as assigning it to session.auth
, but that didn't help either.
Anyone has a clue what could be going on and what I can do to get around this unauthorized access error?