tl;dr: how to use a cheap personal VPN for scraping (code here)
cost: $0 (+ NordVPN free trial or subscription of your choosing)
build time: 15 minutes (MVP)
This post and my work are not sponsored by Nord. I've found it nice to work with …
tl;dr: find spontaenous interactions by scraping tweets with given keywords
cost: $0
build time: 5 minutes (MVP)
Twitter can be a great way to connect with alike minds. But if you're here, you're (probably) not yet a Thought Leader™. Let's walk through a quick way to speed that up …
tl;dr: Use advanced Google Search to find any webpage, emails, info, or secrets
cost: $0
time: 2 minutes
Software engineers have long joked about how much of their job is simply Googling things
Now you can do the same, but for free
Below, I'll cover dorking, the use of …
tl;dr: BeautifulSoup selectors and code snippets
Once you've become familiar with scraping websites with Python, requests, and BeautifulSoup (if not, read this first), you'll want to start creating reusable components to speed build time and improve data reliability.
Below I've included reference snippets for
tl;dr: Using Python requests and BeautifulSoup libraries to scrape data from most data sources
cost: $0
build time: 30 minutes (MVP) / 120 minutes (cleaner v2)
BeautifulSoup allows you to navigate through the raw HTML pulled down from a website (in JS, this is called 'Walking the DOM'). Generally, you …