I have a python script running as service in linux (also as autorun), and it does plenty of output! How can I read this output when the program is already running?
Maybe I could log all the output into a file but then I would have to open and refresh the file all the time when new output is logged!
Well, when it comes to the second paragraph of the question, in the shell you can do:
tail -f logfile.log
which automaticly refreshes when file is updated, so under Linux that's a working solution.
It is also possible to implement tail from the python side, which is basically a continuous reading of it. The code snippet to make this work can be found here:
http://code.activestate.com/recipes/157035-tail-f-in-python/
Additionally, if you use the append mode of file writing instead of the write method you can continuously output.
Scrapy also uses the concept of pipelines which allow for a lot of the same functionality. Here's an example of some scrapy code you might use to do the same thing:
class JsonWriterPipeline(object):
def __init__(self):
self.file = (open(filepath, 'a'))
def process_item(self, item, spider):
self.file.write(json.dumps(dict(item)) + '\n')
return item
