Gathering data from the web

Resources

• HTML Tutorial
• urllib3
• BeautifulSoup Documentation
• Selenium API
• Twitter Search API
• Slides

Lab

• Beautiful soup lab
• Regular expression lab

Using Twitter API Example

• get_tweets.py

Basic HTML example

• Hello.html

Examples from 2014-15 Course Catalog

• Abbreviated CS index page
• CMSC 10100 div
• CS 120s div

Examples from IPDES

• Urban Labs Scraper: IPDES Data Center Scraper
• HTML for the data center page