Prevent exit on exception in Python

Question

I am trying to catch an exception thrown by Selenium. When the exception is thrown I would like to restart the scraping process.

try:
  startScraping(rootPage)
except (InvalidSessionIdException):
  startScraping(rootPage)
finally:
  driver.close()

Above is the code I am using.

The problem I am facing is that when an InvalidSessionIdException occurs the script still stops execution and shows a stacktrace.

Answer 1

the second startScraping(rootPage) (in the except block) is not protected by any try/except block...

If an exception occurs, retrying immediately is probably bound to fail ... again. I would

• catch the exception
• print a meaningful warning
• wait a while
• repeat until it works, or a given number of times with a for loop

like this

import time
nb_retries = 10
for nb_attempts in range(nb_retries):
    try:
      startScraping(rootPage)
      break  # success, exit the loop
    except InvalidSessionIdException as e:
      print("Warning: {}, attempt {}/{}".format(e,nb_attempts+1,nb_retries))
      time.sleep(1)
else:
   # loop went through after 10 attempts and no success
   print("unable to scrape after {} retries".format(nb_retries))

driver.close()

Answer 2

Try this if you want to restart the process and ignoring the exception:

while True:
    try:
      startScraping(rootPage)
      break # after finishing the scraping process
    except (InvalidSessionIdException):
      pass # or print the excepion

driver.close()

as mentioned in the code, you can print the exception or do any other exception handling you may want.