Web Scraping with AutoHotkey is an incredible asset to have at your disposal.  One of my favorite things to do with it is Web Scraping & Pasting.  This page links to the tutorials I have walking through how to get started with Web Scraping in AutoHotkey.  I also have added a lot of example links of work I’ve done (to help stimulate ideas).

Scraping with IE is (mostly) dead however we have lots of alternatives.  In the below video I cover 9 ways to Webscrape with AutoHotkey.  We discuss pros and cons of each approach as well as the skill levels needed to use them.

Web Scraping with AutoHotkey: The ultimate guide for 2023

---

Before watching my videos I highly recommend you familiarize yourself with the DOM (Document Object Model) by watching these videos:

1. What is the Document Object Model Lynda.com video
2. Manipulating the Document Object Model in JavaScript
3. Introduction to Document Object Model

Be sure to grab the Web Scraping Syntax writer

•  ⭐Sponsor: Be a super-star helping us cover costs of new features – $5.99
•  Help fund more Videos & Scripts – $3.00
•  Just give me the download – $0.00

Download IE Web Scraping Syntax Writer Checkout Added to cart

And be sure to get the iWB2 Learner tool

•  ⭐Sponsor: Be a super-star helping us cover costs of new features – $5.99
•  Help fund more Videos & Scripts – $3.00
•  Just give me the download – $0.00

Download iWB2 Learner tool Checkout Added to cart

Check out my review of use various developer tools (and have three versions of the iWB2.)

Also check out my videos on Fiddler for monitoring Network Traffic.  They can really help understand what is being sent from your browser so you know what you need to replicate.

Web Scraping with AutoHotkey and IE

1) Intro Pointer, Get values & Page Navigation
1.5) Intro Troubleshooting & getting correct content from page
2) Intro Set values & clicks / Buttons
3) Intermediate Isolating area & leveraging the DOM/HTML
4) Advanced Dealing with Frames
5) Intro Troubleshooting tips
6) Intermediate Loop over pages & extract data
7) Intermediate Web scraping with AutoHotkey using ClassName
8) Intermediate Web scraping using QuerySelector & QuerySelectorAll
9) Intro Webinar on Intro to Web Scraping
10) Intro Update to Web Scraping Syntax Writer
11) Intermediate Passing Method or Property to COM in a function
12) Advanced Triggering Events on a page
13) Intermediate Saving files / Pictures from a URL / Hyperlink
14) Intermediate Waiting for an Element to Exist / Be visible
15) Intermediate  Extracting data from a table by walking the DOM

Selenium & Cross Browser Web Scraping with AutoHotkey

If you’re interested in Web Scraping with Chrome, FireFox, Opera, Edge, etc. you can check out my page using Selenium with AutoHotkey

Automating Chrome with AHK

1. Getting values from page with AutoHotkey & Chrome
2. Setting values on a page with AHK & Chrome

 

Videos and scripts to Login to Websites:

1. Login to Facebook This first video is much longer & in-depth!  I cover many of the reasons why I pick one method over another.  I also have HellBent sit-in and ask questions so it should be a great starting point for noobs to Web Scraping.
2. Login to Amazon
3. Login to LinkedIn
4. Login to Gmail / Google / YouTube
5. Login to Pinterest
6. Login to Twitter
7. Login to Reddit

---

Examples of work automated via Web Scraping with AutoHotkey

1. Submit StumbleUpon submissions
2. Transfer data from one website/system to another
3. How I exported over 4 million contacts from Lexis Nexis
4. Extract status from SharePoint and email colleagues
5. Select “x” number of items on website form
6. Obtain Behavioral Targeting Data from your own Web Site
7. Determine House status on Real Estate site
8. Extract meta data about videos from website
9. Automating saving invoices on Amazon for taxes
10. Waiting for an element to be visible before clicking (this is using FindText instead of COM)

 

Comparison of Web Scraping Verse API calls