Regular Expressions to automate metrics is an investment everyone should do!

regular expressions to automate metrics

Regular Expressions

Regular Expressions to automate metrics

I’ve written a script that logs into our vendor website and exports the recent email campaigns then breaks them out by region, campaign type, Business Unit, etc. and links them back to our SharePoint site for more information.  The script relies on a regular expression to automate metrics on our long string name and parse-out sections by examining patterns in the Naming convention.

My script first logs into our tool (SilverPop) and performs web scraping to capture the data and dumps it to a local HTML file.  I then use Excel to import the HTML file and start parsing the naming convention with the regular expression.   After all the columns are created I replace some of the short abbreviations with longer, easier to understand, words.   Then I insert functions aggregating the metrics by Type of campaign, Region, Language etc.   After that I break out four regions.  All in all it takes about 10 minutes to run but saves me about 40 hours worth of work.  The below video demonstrates the regular expressions to automate metrics in action

When your IT department fails you Web Scraping with AutoHotkey can save the day!

web scraping with AutoHotkey

One of my clients has over 3,000 videos on their website.  Amazingly nobody in their IT department can provide a holistic file with metadata like: Title, # of views, Length, Format, Tags, etc.  How crazy is that?   After contacting their video hosting company, nobody could (would) share this information!

Web Scraping with AutoHotkey to the rescue!

I wrote web scraping with AutoHotKey script that first got a list of all the videos, then looped over them and extracted various attributes about each video.  Not an ideal way to get this, but the client was manually going to go through this by cutting & pasting from their browse.

Web Scraping with AutoHotKey 103-Leveraging the Document Object Model

Web Scraping with AutoHotKey

Web Scraping with AutoHotKey-Leveraging the Document Object Model

This third video on Web Scraping gets a little advanced and shows how you can leverage the DOM to make extracting data from a webpage much easier and reliable.

Leveraging the Document Object Model (DOM)will take some practice (especially if you’re not familiar with Object oriented coding) but it is well worth it because it greatly reduces the amount of clean-up you have to do after you extract your data.  I used to write some pretty crazy regular expressions to try and clean up my code.  Once I learned how to better navigate the DOM it negated the need for cleaning!

The HTML Document Object Model (DOM)-Tree of Objects

Document Object Model

Video Web Scraping with AutoHotKey Leveraging the DOM plus looping over pages

The syntax for writing the writing the web scraping code can be found on my first post here.  There is also an AutoHotKey forum thread you might wish to review here.