How Stack Overflow fends off scraping bots

Josh Zhang, a staff site reliability engineer at Stack Overflow, tells Ryan and Eira how the Stack Exchange network defends against scraping bots. They also cover the emergence of human botnets, why DDoS attacks have spiked in the last couple of years, and the constant balancing act of protecting sites from attack without inhibiting legitimate users.

Credit: Alexandra Francis

As Josh explains, DDoS attacks aim to take down a website, while bot scrapers try to gather as much data as possible without getting caught.

Josh Zhang is a staff site reliability engineer (SRE) at Stack Overflow. Connect with him on LinkedIn.

ICYMI: In 2022, Josh wrote an article for our blog about how Stack defends itself against DDoS attacks.

Stack Overflow user Serge Ballesta won a Lifeboat badge for answering What does |= mean in c++.

TRANSCRIPT

Add to the discussion