Fixing The Facebook Ad Library (Part I): Scraping Can Save It

What is the ad library?

Double standards

Scraping the ad library

Sounds cool, how do we do it?

How do you scrape pages?
from selenium import webdriversearchDriver = webdriver.Chrome()searchDriver.get(‘')
usernameBox = searchDriver.find_element_by_name(‘email’)usernameBox.send_keys(“your Facebook email”)passwordBox = searchDriver.find_element_by_name(‘pass’)passwordBox.send_keys(“Your Facebook password”)
try:    loginBox = searchDriver.find_element_by_id(‘loginbutton’)except:    loginBox = searchDriver.find_element_by_name(‘login’)
html = searchDriver.page_sourceprint(html)

Two improvements

def scrollDown(driver):    # Get scroll height.
lastHeight = driver.execute_script(“return document.body.scrollHeight”)
while True: # Scroll down to the bottom.
driver.execute_script(“window.scrollTo(0, document.body.scrollHeight);”
# Wait to load the page
# Calculate new scroll height and compare with last scroll height
newHeight = driver.execute_script(“return document.body.scrollHeight”)
# If the browser hasn’t scrolled any more (i.e. it’s reached the end) then stop
if newHeight == lastHeight:

Let’s take stock




Growth marketer — I write about how to run better ads (

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

The Anatomy Behind a $12,000 Website Flip

How to generate traffic to your website

Augmented Reality Can Be Real Gucci

Medium is a money grab — But, most writers aren’t doing it right

Social Snapshot 01.25.17

Media Contact List

media contact list, journalist contact list, medium writer tools, blogging guide, medium tools, media contact list template


How to Build a Thriving Content Marketing Business

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Mack Grenfell

Mack Grenfell

Growth marketer — I write about how to run better ads (

More from Medium

Do you believe in the power of visualization?

Selling AI Platforms to Enterprise

How to generate product names using Open AI

Using Open AI playground to generate unique and catchy product names.

From Start-Up to Successful Online Sales