Python POST form login on website

Question

I'm trying to get this login form working in Python with a work site, so I can do some scraping and whatnot with the site data.

Now, when I print the contents of r.text it's the same HTML as the login page. No progress is made into the site.

It's also worth mentioning that I'm indifferent about using Requests here. I just find the methods and such from urllib to be ugly, and most info I've found on this site has leaned towards using Requests for the above task.

Below is the information I'm working with.

enter image description here

<form action="login.cgi" autocomplete="off" id="frmLogin_4" method="post" name="frmLogin" onsubmit="return Login(1)">
  <input id="tz_offset_5" name="tz_offset" type="hidden">
  <table border="0" cellpadding="2" cellspacing="0" id="table_LoginPage_3">
    <tr>
      <td valign="top">
        <input id="realm_16" name="realm" type="hidden" value="All Users">

        <table border="0" cellpadding="2" cellspacing="0" id="table_LoginPage_6">
          <tr>
            <td>E-Mail Address</td>

            <td>&nbsp;</td>

            <td><input id="username_5" name="username" size="20" type="text"></td>
          </tr>

          <tr>
            <td>Password</td>

            <td>&nbsp;</td>

            <td><input id="password_5" name="password" size="20" type="password"></td>
          </tr>

          <tr>
            <td></td>
          </tr>

          <tr>
            <td colspan="3">&nbsp;</td>
          </tr>

          <tr>
            <td>&nbsp;</td>

            <td>&nbsp;</td>

            <td><input id="btnSubmit_6" name="btnSubmit" type="submit" value="Sign In">&nbsp;</td>
          </tr>
        </table>
      </td>
    </tr>
  </table>
</form>

And below is the Python I'm using at this point.

import requests

url = 'https://<scrubbed>/dana-na/auth/url_default/welcome.cgi'

payload = {
    'username':'<scrubbed>',
    'tz_offset':'-480',
    'realm':'All Users',
    'password':'<scrubbed>',
    'btnSubmit':'Sign In'
}    

session = requests.session()    
r = requests.post(url, payload)
print r.text

amccormack · Accepted Answer · 2014-02-26 03:55:56Z

3

Use the data keyword. From the documentation:

Typically, you want to send some form-encoded data — much like an HTML form. To do this, simply pass a dictionary to the data argument. Your dictionary of data will automatically be form-encoded when the request is made:

payload = {'key1': 'value1', 'key2': 'value2'}
r = requests.post("http://httpbin.org/post", data=payload)
print r.text

Additionally, the HTML you posted suggests the form will actually go to login.cgi and not welcome.cgi. You can see this in the screenshot as well.

Use:

url = 'https://<scrubbed>/dana-na/auth/url_default/login.cgi'

edited Feb 26, 2014 at 3:55

answered Feb 26, 2014 at 3:46

amccormack

14k11 gold badges42 silver badges62 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

David Metcalfe Over a year ago

Brilliant! Changing from welcome to login.cgi fixed it. Thanks a ton!

Collectives™ on Stack Overflow

Python POST form login on website

1 Answer 1

1 Comment

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Linked

Related