retrieving and sending infomation over the web

I found this when hanging around online. To be honest, I am dying to learn web crawling long time ago, can not wait to play with this.

modules involved:

This post is very easy to follow with basic familarity of Python, the difficulty of web crawling is lots of, if not all, websites follow HTML specification exactly, there are often errors here and there if you try to parse useful bits of infomation out there, and here comes BeautifulSoup which handle most of these for us.

The second section discuss sending email with Python. One thing need to keep in mind is check your internet connection first if your code refuse to work :-)- I have complete the spec of No.5, and below are some commands I find useful:

  • execfile('script.py') #load script into console
  • python -i script.py #execute script then come to Python console, ‘i’ short for ‘interactive’
  • sometimes you need to insert a time stamp into you file, and time.ctime() is probably the simplest solution
    source code is here