xmlHTTPRequest vs. WinHTTPRequest- What killer advantages does the xmlRequest have?

xmlHTTPRequest vs. WinHTTPRequestIn this video I walk through both the xmlHTTPRequest and the WinHTTPRequest and I compare the xmlHTTPRequest vs. WinHTTPRequest.  As I mention in the video, Jackie Sztuk and I have a great  AutoHotkey webinar on Intro to API calls.  I also have several examples on my API page.

I also mention using Fiddler to monitor the browser traffic and grab your cookies / headers.

xmlHTTPRequest vs. WinHTTPRequest tutorial


Here’s the syntax I used for the WinHTTPRequest example

WinHTTPRequest syntax

And here’s the corresponding two XML API calls I demonstrated in the video

xmlHTTPRequest example using Msxml2.XMLHTTP COM object

Here’s the example connecting to the IE page and sending the API request with the xmlHTTPRequest()

01: Chrome and AutoHotkey- Connecting to Chrome with AutoHotkey | Amazing deep-dive into geeky things

Connecting to Chrome with AutoHotkeyConnecting to Chrome with AutoHotkey

This discussion wasn’t meant to be shared.  GeekDude was giving me some background on how we’re connecting to Chrome.  It is a bit “advanced” but some really good background info (especially understanding what a socket  verse WebSocket is).  Below is the video and my transcript-ed notes from the discussion

donate to GeekDudeIf you’re loving this, please consider donating to GeekDude!

Connecting to Chrome with AutoHotkey

1:50        Why starting with remote debugging port

1:41        Pages in debugging environment

2:18        Other browser automation tools like Selenium realized this is a great way to connect

2:53        The debugging tools like Chrome, Selenium, FireFox all adopted this same approach

4:00        Devtools protocol  https://chromedevtools.github.io/devtools-protocol/

4:09        Can I get the protocol as JSON?  If you’ve set –remote-debugging-port=9222 with Chrome, the complete protocol version it speaks is available at localhost:9222/json/protocol (remember to close all instances of Chrome before launching in debug mode)

4:30        The JSON string talks about everything you can do with the protocol

4:55        If you browse to this JSON page,  http://127.0.0.1:9222/json  Chrome will show you all the debugable pages.  Tabs, Plugins, etc.

5:44        In json Look for webSocketDebuggerUrl and pick a “page”.  That will allow you to automate it

6:40        Iframe example with Google Doodle URL.  This will give you just the iFrame.   Get the ” devtoolsFrontendUrl” path then concatenate with your ip& port (http://127.0.0.1:9222 )   for example my hangouts was:  http://127.0.0.1:9222/devtools/inspector.html?ws=127.0.0.1:9222/devtools/page/002FA737EFE330712C084757D33748F6

7:23        All you see in the debugger is from that iFrame (because we opened that iFrame directly)

9:06        other things marked as “pages”.  Long strings are probably extensions where someone didn’t fill out their info correctly

10:00     Automate plugins like lastpass.  It’s not documented yet, but you can see how to connect to it

11:00     When create instance of Chrome, it launches the Chrome browser and trys to get a specific debug port and then it saves that number for that instance.

11:37     We could have used the number

11:49     When creating other instances (GetPage() it takes that websocket debugger URL and passes it to the class “page” (in Chrome.ahk).

12:20     If there is a class in future versions of Chrome.ahk, he’ll probably only have the page class.  Because everything being done before you connect to that page is not live.      You have a live connection to the browser.  Everything up to this point wasn’t a “live” connection.  Once you have a connection to the page, it needs to be updated…

12:50     What is a websocket?

13:19     A socket is when you open a connection to another machine and you can send data to it and get data back.  It stays open and you can continue to transfer data back and forth

13:20     A webRequest is where you open a connection to a machine, you ask for a resources, it can wait and, when you get that resources back, you’re “done” an the connection is closed

13:40     Websockets bridge the two.  You start by sending a webrequest that says you want to open a websocket connection so that rather than a get/post winhttprequest, this is a special kind of request.  It “upgrades” that connection to a websocket connection.  From there it is much more similar to a regular socket.  You can send data back and forth.

What are WebSockets

14:29     This has been difficult to do from AutoHotkey because websockets were designed with a lot of abstraction to make things easier for the javascript developer.   A socket is much more loosey-goosey in that you send some bytes, they probably get there, probably not get there all at the same time, you fill up the buffer, occasionally flush the buffer, etc.  Websockets handle all of this for you! You send a “message” and it gets encapsulated.  The browser only exposes to you full messages.

So you don’t have to deal with text encoding, waiting for the full bytes, it all gets handled automatically.  That process takes a lot of extra code.  Even if you ignore the Secure sockets layer (SSL) writing all of that encryption code in AutoHotkey would be borderline insanity.  So it’s just not available.

15:52     That’s why when GeekDude  wrote Chrome.ahk and Discord.ahk, they both just create an instance of IE in the background and use ActiveX / COM to handle the WebSocket code.   This is fast but it is part of the instability.  It works great for the most part, but sometimes it just breaks down.

17:13     If IE dies, are we going to need to find another way?  GeekDude  thinks IE might never go away however he heard about websockets CAPI WebSocket Protocol Component API Functions for doing websockets.   This could be our way to create the WebSocket connection.

17:55     There’s a WebSocketCreateClientHandle function.  He’s not sure what it means, but it looks like a DLL compatible API call.  Hopefully we can use this to ditch IE.  Taking this approach will make it strange to implement Teadrinker’s solution.

 

AutoHotkey Web service API syntax writer- Automate writing your WinHTTP Requests with my syntax writer

AutoHotkey API syntax writer

In this video I walk through how to use my AutoHotkey API Syntax Writer.  If you’re new to APIs, be sure to check out our Webinar on API / Web Services or this page for a lot of examples of connecting to API services.

Please remember that, after launching the script, you’ll need to hold down the control key and left mouse-click to activate the menu options.AutoHotkey Merchandise-White Stress ball

Here is the video walking through how to use the tool.

 

AutoHotkey Webinar- Intro to API calls with AutoHotkey

Intro to API calls with AutoHotkeyIn this AutoHotkey Webinar we cover an Intro to API calls with AutoHotkey.

Video Hour 1: High-level:

  • What is a Webservice API call
  • Web Scraping vs. WebServices / API
  • List of some “fun” WebService / APIs as well as links to resources to tens-of-thousands!
  • Differences between a Get vs. Post request
  • High-level look at oAuth1 vs. oAuth2
  • Resources
    • HTTP: Intro to HyperText Protocol, Types of Requests, oAuth, Parsing XML, XPath, JSON
    • AutoHotkey resource: Syntax writer, winHTTPrequest, Msxml2 vs. WinHTTPrequest, Parsing XML & JSON
  • Basic examples of API requests (compared using browser and winHTTP request)

Video Hour 2: Coding and Q&A

  • Played with several APIs
  • Discussed & demonstrated reverse-engineering API calls from a website
  • Passing key-value pairs (and using a function to keep it organized)
  • Reviewed additional APIs
  • Q&A

Script Highlight: Select text and “pretty” hyperlink text

The below script demonstrates how you can use AHK to automate highlighting text on a page and then, using the winClip library, constructing a “pretty” html link.  Here are links to the GetActiveBrowserURL and WinClipAPI / WinClip

Intro to API calls with AutoHotkey


What is a Webservice API? (Application Programming Interface?)

API calls with AutoHotkey


Examples of Webservice / API calls with AutoHotkey

  • Application / Software querying products for sale on Amazon.com
  • App on your phone getting latest Weather
  • Database pulling updated sales report
  • Using your Tablet to Select movies to watch on Netflix
  • DropBox application syncing files between your computer & cloud
  • Google places search

APIs are becoming increasingly available!

…Since 2005, we’ve seen APIs grow from a curiosity, to a trend, and now to the point where APIs are core to many businesses. APIs have provided tremendous value to countless organizations and developers, which is reflected in their continued growth.   Source: Programmable Web

API calls with AutoHotkey

 

 


Main Differences between WebService / API call & Web Browser

API calls with AutoHotkey


When to use API verse Web Scraping?

API calls with AutoHotkey


Some Webservices / APIs

Some of the Amazing APIs out there…

Programmable Web – list of thousands of APIs as well as a great, six-part, article reviewing APIs

Short-list of APIs open to public on Wikipedia,   Google APIs– Over 100 APIs @ Google; Yahoo APIs

Social Sites : Twitter, Pinterest, Google+, YouTube, Facebook, LinkedIn, reddit, StumbleUpon, WordPress, Instagram

Contact / Business Lookup: ClearBit, Pipl, FullContact, Dunn & Bradstreet , Foursquare, Google Places, Yelp, GeoNames

Weather: Weather Underground, Yahoo Weather, World Air

Finance: Google Finance, Yahoo Finance, PayPal, Stripe, XigniteRealTime, Ally Invest

Government: A bunch of Government APIs, Cloud.Gov, US Census, 18F.gsa.gov

Additional: Wikipedia, Amazon Product, email to text, SmartSheet,SurveyGizmo, Bitly, SnapOCR, PasteBin, DropBox, Zoho, Zillow, imugr, Mailgun, MailChimp,  Microsoft: Graph, MS Office: Excel  Word  SharePoint  OneNote Outlook Yammer PowerPoint


Break-down of a REST API request via COM object

  • Create COM object
  • Open Endpoint (w/ parameters & Authentication if GET request)
  • Set RequestHeader(s)
  • Send (w/”payload & Authentication” for POST requests)
  • Get response (body or text)

Main differences of oAuth1 verse oAuth2

Unfortunately there is no “standard” implementation of oAuth1 or oAuth2 however, at a high-level, here are some of the main conceptual differences:

oAuth1:

  • Need a Key & Token from Webservice API (typically different than your username and password)
  • You use your Key & Token in your API call
  • oAuth1 is less secure and, generally, being phased out
  • While being phased out often the “developers” (us) can use oAuth1 for development of the “app”

oAuth2:

  • Need a Key and Token (same as oAuth1) however you use the Key & Token and some other parameters to perform a “handshake” which returns a secure token which typically times-out in seconds / minutes/hours
  • Your token is restricted to the level of your account (or what has been authorized)
  • The secure token is what is shared with your actual endpoint. (this allows other Social sites (like LinkedIn, Facebook, etc.) to assist your login but not have your username/password to the endpoint

HTTP and AutoHotkey Resources

HTTP Protocol & General Tutorials

API calls with AutoHotkeyAutoHotkey specific



Let’s make some API calls with AutoHotkey!

Weather Underground

  1. Dallas forecast in XML
  2. Dallas forecast in JSON
  3. Using WinHttpRequest
  4. Example with browser

Yahoo business

  1. Pizza restaurants in zip code 75019 in XML
  2. Pizza restaurants in zip code 75019 in JSON
  3. Using WinHTTPRequest

Intro to API calls with AutoHotkey