• Intro to AutoHotkey HotStrings with AutoHotkey Intermediate AutoHotkey GUIs are Easy with AutoHotkey Intro to DOS & AutoHotkey

04: Automating Chrome to Set Text & Click a button

Automating Chrome with AutoHotkey

In the fourth session with GeekDude we look at out to Chrome and AutoHotkeyautomate setting text in a search field and then hitting the button to submit the search.

Automating Chrome to Set Text & Click a button


donate to GeekDudeIf you’re loving this, please consider donating to GeekDude!

AutoHotkey script for Automating Chrome to Set Text & Click a button


#Include   ;Remember to put Chrome in your library folder

#SingleInstance,Force
;**************************************
page:=Chrome.GetPageByTitle("AutoHotkey Community","contains") ;This will connect to the second index of a specific tab
If !IsObject(page){
MsgBox % "That wasn' t object / the page wasn't found"
ExitApp
}
page.Evaluate("document.querySelector('#keywords').value ='Chrome.ahk'")
Variable =document.querySelector('#keywords').value ='Chrome.ahk'
page.Evaluate(Variable)
var:="duh"
page.Evaluate("document.querySelector('#keywords').value ='" var "'")
page.Evaluate("document.querySelector('#search > fieldset > button').value ='Chrome.ahk'")

Notes for Automating Chrome to Set Text & Click a button

00:36     Go to AutoHotkey.com/boards/

00:44     Connect to tab using Chrome.GetPageByTitle(“AutoHotkey Community”) ;the default matchtype is “starts with”

01:23     Look at page structure using right-click and Inspect.  This opends Devtools with that element selected.

01:46     It has an ID of “keywords”, copy js path.  Which will give you queryselector(“#keywords”)

02:26     Use the .value to set some text in that box.

03:00     page.Evaluate(“document.querySelector(‘#keywords’).value =’Chrome.ahk'”)

04:01     Make sure inside the JavaScript you use the “=”, not “:=”

04:15     Some people don’t want to have to learn JavaScript.  When using Chrome, you’re going to have to learn JavaScript.

04:56     When using Chrome.ahk, we’re injecting JavaScript.  So best to learn

05:54     The button is right next to the input.  You can go back to the page and right-click the button, then hit Inspect

06:13     Test the new js path.  Instead of using .value, use .click

06:42     Test in Chrome developer tool

07:18     When running an Evaluate method, it waits for the previous Evaluate to finish (so no need to sleep between them).

07:44     If you run into a problem where you think it is happening too quickly, check the forum for some solutions

08:40     Sometimes what you want to input won’t always be a static string.  If you’re trying to reference a variable, you need to use the expression syntax.  In an expression, you’re not just assigning text, you’re doing math or making function calls.

Variable =document.querySelector(‘#keywords’).value =’Chrome.ahk’

page.Evaluate(Variable)

page.Evaluate(“document.querySelector(”#keywords ‘).value ='” variablevar:=”duh”

page.Evaluate(“document.querySelector(‘#keywords’).value ='” var “‘”) “‘”)

10:48     This works because AutoHotkey splits everything up on a given line.   First is a name of a function, then says this is inside the function, then this is text inside a function.  Then builds from left to right as to the string that will be used.

12:15     AutoHotkey proceeds left to right when evaluating an expression

12:40     when you use := you’re in expression assignment mode.

13:25     With just single = you’re in plain-text mode.  It reads it as text

15:00     When automating a site, you don’t know what kind of buffer’s they have to prevent scraping / botting.

15:49     When you start automating, you might start seeing Captcha’s everywhere

16:04     Sites get really good at looking like a normal site to a user, but looking like an impenetrable fortress to code

16:36     If your variable contains a single quote or other special charachters, JavaScript will interpret it as code instead of text.

17:13     JavaScript string escape sequence will replace characters with special escape sequences

Not mentioned in Video but GeekDude wrote me after

You can escape JavaScript code using Coco’s JSON library does actually do that escaping that we discussed when talking about putting data on the page. The syntax for invoking it looks like this:

variable = 123`r`n456’quote”quote

page.Evaluate(“document.querySelector(‘#whatever’).value = ” Chrome.Jxon_Dump(variable))

The dump function will automatically escape anything that needs escaped and add quotes to anything that needs quotes.

 

 

03: How to Automate Chrome to Navigate to a page and Set text

Chrome and AutoHotkey

Automating Chrome with AutoHotkey

In this session with GeekDude we cover how to navigate to a page, use JavaScript, and set text on a page / element.

donate to GeekDudeIf you’re loving this, please consider donating to GeekDude!

Script from Call


 

My notes from the session

00:30     Navigate to

00:50     connect to page

01:45     Call the DevTools function Page.Navigate  “https://p.ahkscript.org/”)

02:38     you need to call it by using page.Call(“Page.navigate”,{“url”:“https://p.ahkscript.org/”})

03:00     Look at parameters of Page.Navigate to understand what you need to pass

04:07     You can use page.WaitForLoad() to make sure the page loads (but it isn’t always needed).  Sometimes it waits too long or not long enough.  Check the forum thread for possible help

05:00     You might want to adapt some of their code to update GeekDude’s Chrome class

06:10     Make sure your page grabs the title of the right page (as it can update)

06:40     Put text into page element using JavaScript.  The input field on p.ahkscript.org isn’t a normal edit field.  Many sites have more-advanced field types

07:34     Inspect the page source.  This website imports a script called ace.js

<script src=”https://cdnjs.cloudflare.com/ajax/libs/ace/1.2.7/ace.js” type=”text/javascript” charset=”utf-8″></script>

08:30     It has it’s own JavaScript API.  So when you want to talk to an Ace control, you need to talk in it’s language.  You’ll need to evaluate your site to see what it is using…  Always look for how to work with it using JavaScript.  When you’re using Chrome.ahk everything you’re doing is running through JavaScript!

8:57        You’re using AutoHotkey to run the JavaScript to drive Chrome.

9:00        In https://p.ahkscript.org/index.js you can see how that page talks to Ace. It could give you information on how to do it.  Mimic what the page itself is doing.   Not all sites are as easy as this to see what does

10:00     In this example, It creates an Editor variable and calls all of the editor commands on that.

10:23     Look back to Ace.js documentation you can see Editor is a section of the documentation.  This shows all the different methods to communicate with an Ace control.

10:47     Since we want to set text look for one that does that.  We’ll use Insert  function (Inserts text into wherever the cursor is pointing.)  Insert(String text)

11:15     In the  pastebin code’s index.js we see everything is called with editor dot (editor.) method name

11:25     We’ll prototype our JavaScript code from the Chrome developer tools because those give better immediate feedback (debugging info) of what went wrong.  Once we have a working line of JavaScript we’ll put that in AutoHotkey.

11:30     Go to Console within the Chrome debugging view (Control+Shift+i)

11:48     Look for the variable Editor by typing Editor.  It should respond showing you that it exists and is available

11:56     Auto-Assist works.  Try typing Editor.Insert (because insert is a method) .  Give it some text to test.

editor.insert(“hello world”)

12:17    The text should show up on the pastebin page.  😊

12:28     Take it over to your AutoHotkey code and try it from there using Page.Evaluate

12:42     The Double quotes are a problem because ahk and JavaScript both use them.

13:00     You could use the legacy assignment to set it.  But this gives us 2 lines of code

13:40     Thankfully JavaScript can use single quotes  most of the time.

page.Evaluate(“editor.insert(‘hello world’)”)

14:00     The Evaluate Method- that’s saying “evaluate the JavaScript” as it is using the same functionality as putting JavaScript in the Chrome development tools console and hitting Enter.  It runs it through the same interface.  Evaluate is short for “Take this JavaScript, run it on the page, and return the result.

14:53     Maybe your JavaScript has a ” that you want .   AutoHotkey requires you to double-up the quotes  “” (basically it escapes the second one).  AutoHotkey will reduce it from two quotes to one quote.

15:30     Maybe the page your using is simpler.  A lot of places have very simple pages.  They’ll just have standard input control which you can access with standard JavaScript.  Another site is mytextarea.net

16:12     Inspect the code for mytextarea.net  and you’ll see it is literally just a textbox

16:20     Let’s navigate to that page.  You’ll see we have to find a different way to connect as every page is unique.

17:16     When you’re writing JavaScript to a page you don’t know much about, one of the biggest problems you’re going to have is how do I get it to act on just this part of the page, this element, this box?

17:31     When in DevTools console, right click on the element you want, then Copy, and select Copy JS path (Selector is also helpful)

18:00     This returns the JavaScript code that that can be used to access that element (it is the Path to it)

18:00     So use that path, then we want to get it’s value

document.querySelector(“body > textarea”).value

18:48     this uses QuerySelector which returns the first instance.  QuerySelectorAll returns an array.

20:14     As long as you copy js Path you’ll probably get something useful (not sure if it is the DOM or how it is decided)

20:20     Any tools you can use like iWB2 learner tool for Chrome- No, GeekDude not aware

21:00     Here we’ve access a value.  Let’s set a new value

document.querySelector(“body > textarea”).value =’fruit’

21:26     Now take this from the Devtools and put it into AutoHotkey.  Make sure you replace double quotes with single quotes  page.Evaluate(“document.querySelector(‘body > textarea’).value =’taco'”)

21:30     Make sure the title / URL is the correct one

 

 

 

 

 

02: Connecting & starting a page in Chrome and AutoHotkey

Chrome and AutoHotkeyConnecting & starting a page in Chrome and AutoHotkey

In this session GeekDude shows me several ways to connect to an existing Chrome page.


donate to GeekDudeIf you’re loving this, please consider donating to GeekDude!

Connecting & starting a page in Chrome and AutoHotkey

Connect to tab and page

1:00        Chrome.ahk was developed as a way to connect to Chrome w/o any external tools (unlike Selenium)

1:20        Don’t use Selenium, Use the Chrome.ahk class from GeekDude

1:43        All you need is Chrome and AutoHotkey.  Selenium install can be difficult

2:40        Chrome.ahk talks to Chrome using Chrome’s developer tools protocol.  Meaning if you open-up the page inspector (Control+Shift+I) or “inspect” an element. Anything you can do with the page inspector / debugger, you can do with AutoHotkey on Chrome.

3:20        Because AutoHotkey is external to Chrome you have to start Chrome with a special command line flag.  chrome.exe –remote-debugging-port=9222 . The easiest way to do this is to find your shortcut for launching Chrome (taskbar, Start menu, etc.) go to the shortcut properties and in the “target” add above code.

4:28        Every time you open Chrome, it will allow you to open the devtools window but instead of using the Control+Shift+I shortcut, it uses a network connection which allows you to connect to your Chrome window.

5:07        GeekDude thinks this is a “local” connection (meaning localhost) so it is probably just your computer.

6:16        Close out of any Chrome windows you have open. Be sure you inspect your system tray, Chrome might stay running in the background as will hangouts.  Both of these need to be closed!!!)

6:20        When you reopen, Chrome will be listening for the debugging connections (listening for AutoHotkey to speak to it).

7:10        Create a new AutoHotkey script with your favorite editor.  GeekDude  uses his CodeQuickTester AHK script

7:35        #include your Chrome.ahk class / library.  (I.e.   #Include C:\AutoHotkey_L\Lib\Chrome.ahk)

8:00        Many different ways to start the Chrome Class.  This video we’re connecting to an existing window

8:19        Create a variable Page:=Chrome.GetPage()

9:15        When you’re connecting to Chrome, Chrome separates different items (tabs, Extensions, etc.) into something called a “page”.  Each item has a different logical page.

9:47        Here GeekDude explains how a method in a class works.  Because Chrome is a class and GetPage() is a method in that class, you need to use both to use it  (I was confused because there was no “chrome” variable.    This is called a static Method or static function.  You don’t have to do anything with it, you can just call it.

10:45     Check to confirm the class did find a page.  Use msgbox % IsObject(page).  You should get  1 which means it successfully connected to a page.

12:35     Chrome library on Github has a lot of examples.  We borrow from it to get the page.

13:08     page.Evaluate(“alert(‘hi’);”) ;Call Evaluate on the page variable.

14:01     A method is just a function that acts on an object / class.  You can use Method and function interchangeably here.

14:20     The Evaluate function is sending JavaScript to the debugger console

14:46     The Popup worked, but it was on the “wrong” page.  We wanted the New Tab page

15:02     Go look inside Chrome.ahk and you’ll see several ways to connect to a page.

15:24     GetPage() function but also GetPageByTitle(), GetPageByURL() and GetPageBy()

15:40     The other GetPageBy() methods are less useful for automation.  If you can’t use GetPageByTitle() or GetPageByURL then reach out to him via Discord, email, his website https://www.philipt.net/.

16:31     Change from GetPage to use GetPageByTitle()

17:00     If you have more than 1 tab with the same name, it looks like it gets them in the order they are in (this is in flux)

17:50     Add parameters to the GetPageByTitle to tell it which tab you connect to.  page:=Chrome.GetPageByTitle(“New Tab”, “startswith”,2)

18:49     GetPageByURL – make sure you click several times in Chrome to get the full URL

19:20     You should only need to connect to the page once.  If you connect when it starts, just keep the pointer to that page for the rest of the time.  If you restart your script, you’ll need to reconnect to that page if the title or URL changed

19:52     Currently using the “startswith” matching option.  There is also “contains”  (it can be anywhere in the URL)  e.g. https://www.reddit.com/r/google/ will still be found when searching for “google”.

20:45     “exact” matching is case insensitive.

21:00     Use the various matching to adapt to your needs.

22:11     Change the program to only alert us when it is not an object (!)

 

 

 

01: Chrome and AutoHotkey- Connecting to Chrome with AutoHotkey | Amazing deep-dive into geeky things

Connecting to Chrome with AutoHotkeyConnecting to Chrome with AutoHotkey

This discussion wasn’t meant to be shared.  GeekDude was giving me some background on how we’re connecting to Chrome.  It is a bit “advanced” but some really good background info (especially understanding what a socket  verse WebSocket is).  Below is the video and my transcript-ed notes from the discussion

donate to GeekDudeIf you’re loving this, please consider donating to GeekDude!

Connecting to Chrome with AutoHotkey

1:50        Why starting with remote debugging port

1:41        Pages in debugging environment

2:18        Other browser automation tools like Selenium realized this is a great way to connect

2:53        The debugging tools like Chrome, Selenium, FireFox all adopted this same approach

4:00        Devtools protocol  https://chromedevtools.github.io/devtools-protocol/

4:09        Can I get the protocol as JSON?  If you’ve set –remote-debugging-port=9222 with Chrome, the complete protocol version it speaks is available at localhost:9222/json/protocol (remember to close all instances of Chrome before launching in debug mode)

4:30        The JSON string talks about everything you can do with the protocol

4:55        If you browse to this JSON page,  http://127.0.0.1:9222/json  Chrome will show you all the debugable pages.  Tabs, Plugins, etc.

5:44        In json Look for webSocketDebuggerUrl and pick a “page”.  That will allow you to automate it

6:40        Iframe example with Google Doodle URL.  This will give you just the iFrame.   Get the ” devtoolsFrontendUrl” path then concatenate with your ip& port (http://127.0.0.1:9222 )   for example my hangouts was:  http://127.0.0.1:9222/devtools/inspector.html?ws=127.0.0.1:9222/devtools/page/002FA737EFE330712C084757D33748F6

7:23        All you see in the debugger is from that iFrame (because we opened that iFrame directly)

9:06        other things marked as “pages”.  Long strings are probably extensions where someone didn’t fill out their info correctly

10:00     Automate plugins like lastpass.  It’s not documented yet, but you can see how to connect to it

11:00     When create instance of Chrome, it launches the Chrome browser and trys to get a specific debug port and then it saves that number for that instance.

11:37     We could have used the number

11:49     When creating other instances (GetPage() it takes that websocket debugger URL and passes it to the class “page” (in Chrome.ahk).

12:20     If there is a class in future versions of Chrome.ahk, he’ll probably only have the page class.  Because everything being done before you connect to that page is not live.      You have a live connection to the browser.  Everything up to this point wasn’t a “live” connection.  Once you have a connection to the page, it needs to be updated…

12:50     What is a websocket?

13:19     A socket is when you open a connection to another machine and you can send data to it and get data back.  It stays open and you can continue to transfer data back and forth

13:20     A webRequest is where you open a connection to a machine, you ask for a resources, it can wait and, when you get that resources back, you’re “done” an the connection is closed

13:40     Websockets bridge the two.  You start by sending a webrequest that says you want to open a websocket connection so that rather than a get/post winhttprequest, this is a special kind of request.  It “upgrades” that connection to a websocket connection.  From there it is much more similar to a regular socket.  You can send data back and forth.

What are WebSockets

14:29     This has been difficult to do from AutoHotkey because websockets were designed with a lot of abstraction to make things easier for the javascript developer.   A socket is much more loosey-goosey in that you send some bytes, they probably get there, probably not get there all at the same time, you fill up the buffer, occasionally flush the buffer, etc.  Websockets handle all of this for you! You send a “message” and it gets encapsulated.  The browser only exposes to you full messages.

So you don’t have to deal with text encoding, waiting for the full bytes, it all gets handled automatically.  That process takes a lot of extra code.  Even if you ignore the Secure sockets layer (SSL) writing all of that encryption code in AutoHotkey would be borderline insanity.  So it’s just not available.

15:52     That’s why when GeekDude  wrote Chrome.ahk and Discord.ahk, they both just create an instance of IE in the background and use ActiveX / COM to handle the WebSocket code.   This is fast but it is part of the instability.  It works great for the most part, but sometimes it just breaks down.

17:13     If IE dies, are we going to need to find another way?  GeekDude  thinks IE might never go away however he heard about websockets CAPI WebSocket Protocol Component API Functions for doing websockets.   This could be our way to create the WebSocket connection.

17:55     There’s a WebSocketCreateClientHandle function.  He’s not sure what it means, but it looks like a DLL compatible API call.  Hopefully we can use this to ditch IE.  Taking this approach will make it strange to implement Teadrinker’s solution.

 

  • Intro to AutoHotkey HotStrings with AutoHotkey Intermediate AutoHotkey GUIs are Easy with AutoHotkey Intro to DOS & AutoHotkey