Before you keep reading...

Runestone Academy can only continue if we get support from individuals like you. As a student you are well aware of the high cost of textbooks. Our mission is to provide great books to you for free, but we ask that you consider a $10 donation, more if you can or less if $10 is a burden.

Support Runestone Academy Today

Before you keep reading...

Making great stuff takes time and $$. If you appreciate the book you are reading now and want to keep quality materials free for other students please consider a donation to Runestone Academy. We ask that you consider a $10 donation, but if you can give more thats great, if $10 is too much for your budget we would be happy with whatever you can afford as a show of support.

Support Runestone Academy Today

14.3. Plan 2: Get a soup from a URL

14.3.1. Plan 2: Example

The first step in web scraping is getting information from a webpage. To use the BeautifulSoup web scraping library, we have to put the webpage into something called a soup.

Here is the code for getting a soup from the Cottage Inn location page.

Goal: Get a soup from one webpage
# Load libraries for web scraping
from bs4 import BeautifulSoup
import requests
# Get a soup from a URL
url = 'https://cottageinn.com/pick-a-location/'
r = requests.get(url)
soup = BeautifulSoup(r.content, 'html.parser')

14.3.2. Plan 2: When to use this plan

Use this plan when you want to scrape one webpage.

14.3.3. Plan 2: How to use this plan

Replace the URL with the URL of the website you want to scrape.

A URL is a web address, like you see in your web browser. It should be complete (starting with http:// or https://). In this plan, a URL should be surrounded by quotes (' ').

14.3.4. Plan 2: Exercises

csp-10-2-1: If you wanted to get a soup from the MDen homepage instead of the Cottage Inn location page, which part(s) of the code below would you change? Click on those part(s) of the code.Check out the example of this plan above to identify the area that should be changed.

# Load libraries for web scraping
from bs4 import BeautifulSoup
import requests

# Get a soup from a URL
url = 'https://cottageinn.com/pick-a-location/'
r = requests.get(url)
soup = BeautifulSoup(r.content, 'html.parser'):

csp-10-2-2: Fill in the plan in order to get a soup from the University of Michigan wikipedia page.

# Load libraries for web scraping

from bs4 import BeautifulSoup

import requests

# Get a soup from a URL

url =

r = requests.get(url)

soup = BeautifulSoup(r.content, 'html.parser')

Note

Click here to go back to the Cottage Inn example

You have attempted of activities on this page