Instacart Web Scraping: Extracting Grocery and Delivery Data

1. Required Tools and Libraries

To scrape Instacart data, you will need:

2. Install the Required Libraries

Use the following commands to install the necessary libraries:

3. Choose a Browser Driver

For dynamic content rendering, use ChromeDriver or GeckoDriver with Selenium.

Step 1: Analyzing the Instacart Website Structure

Before scraping, explore the HTML structure of the Instacart website:

• • Product names
• • Prices and discounts
• • Delivery fees and times
• • Categories and product descriptions

Step 2: Extracting Static Instacart Data Using BeautifulSoup

import requests
from bs4 import BeautifulSoup

url = "https://www.instacart.com"
headers = {"User-Agent": "Mozilla/5.0"}
response = requests.get(url, headers=headers)
soup = BeautifulSoup(response.content, "html.parser")

# Example: Extract product names
titles = soup.find_all('h2', class_='product-title')
for title in titles:
    print(title.text)

Step 3: Extracting Dynamic Instacart Data Using Selenium

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.service import Service
import time

# Set up Selenium driver
service = Service("/path/to/chromedriver")
driver = webdriver.Chrome(service=service)

# Open Instacart homepage
driver.get("https://www.instacart.com")
time.sleep(5)  # Allow time for JavaScript to load

# Extract product names
titles = driver.find_elements(By.CLASS_NAME, "product-title")
for title in titles:
    print(title.text)

driver.quit()

Step 4: Extracting Pricing and Delivery Data

# Navigate to product page
driver.get("https://www.instacart.com/store/product-page")
time.sleep(5)

# Extract item name and price
item_name = driver.find_element(By.CLASS_NAME, "item-title").text
price = driver.find_element(By.CLASS_NAME, "item-price").text

print(f"Product: {item_name}, Price: {price}")
driver.quit()

Step 5: Storing the Extracted Data

import pandas as pd

data = {"Product": ["Milk", "Bread"], "Price": ["$3.49", "$2.99"]}
df = pd.DataFrame(data)
df.to_csv("instacart_data.csv", index=False)