===========================
Gathering data from the web
===========================

Resources
---------

- `HTML Tutorial <https://www.w3schools.com/html/default.asp>`_
- `urllib3 <https://urllib3.readthedocs.io/en/latest/>`_
- `BeautifulSoup Documentation <https://www.crummy.com/software/BeautifulSoup/bs4/doc/>`_
- `Selenium API <https://seleniumhq.github.io/selenium/docs/api/py/api.html>`_
- `Twitter Search API <https://pypi.python.org/pypi/TwitterSearch>`_
- :download:`Slides <./slides.pdf>`

Lab
---

- `Beautiful soup lab <../../labs/lab1/index.html>`_
- `Regular expression lab <../../labs/lab2/index.html>`_

Using Twitter API Example
-------------------------

- :download:`get_tweets.py <./get_tweets.py>`


Basic HTML example
------------------

- :download:`Hello.html <./hello.html>`


Examples from 2014-15 Course Catalog
------------------------------------

- :download:`Abbreviated CS index page <./cs-index.html>`
- :download:`CMSC 10100 div <./101-div.html>`
- :download:`CS 120s div <./120s-div.html>`

Examples from IPDES
-------------------

- `Urban Labs Scraper: IPDES Data Center Scraper <https://github.com/UrbanInstitute/ipeds-scraper>`_
- :download:`HTML for the data center page  <./ipeds-datacenter.html>`