AutoHotkey webinar: Deep-Dive into File Encoding

Two women smiling and looking at a computer screen together in a bright office, one pointing at the monitor.In this AutoHotkey Webinar took a deep-dive into File Encoding by our guest speaker Jean Lalonde (author of Quick Access Popup)

Video Hour 1:  High Level:

  • Why File encoding matters
  • Pros/Cons of each type of encoding
  • Tools like the File Encoding Lab to help determine a file’s encoding
  • How to set File Encoding in AutoHotkey

Video Hour 2: Coding and Q&A

Script Highlight: BarChart by Learning One

Bar chart with yellow bars showing Jackie 24, Jean 22, Joe 27 and LearningOne 23 as a sample of the BarChart script

(Here is a link to BarChart scripts demonstrated during the webinar)


1) File Enconding in AutoHotkey

(Here is a link to all of Jean Lalonde’s files)
  • Edit and run the AHK script “FileRead.ahk”
  • Load “Demo-UTF-16.txt” with default encoding
  • Load “Demo-UTF-16-No_BOM.txt” with default encoding  -> problem!
  • Load “Demo-UTF-16-No_BOM.txt” with UTF-16-RAW encoding  -> OK!
  • Try the other file enconding available in AHK

2) File Encoding in DOS (code pages)

  • Run the batch file “Type Box.bat” in DOS console (under “Tutorial“ folder)
  • See this ASCII/ANSI file displayed with code pages 1252 (default ASCI/ANSI) and 437

3) Load the File Encoding Lab

  • Run the AHK script “File Encoding Lab.ahk”
  • Loaded bu default “ASCII.txt” (detected CP1252, 7-bit chars)
  • File Encoding Lab tour
    • Binary display on the left side / Normal AHK display on the right
    • Click on the file name to see with Notepad
    • Encoded detected and use the dropdown list to reload with another encoding
  • Load “ANSI.txt” (detected CP1252, 8-bit chars)
  • Load “Box-CP437.txt” (detected CP1252, 8-bit chars)
    • Which encoding will display a box?
  • Load “UTF-8.txt” (detected UTF-8)
    • see BOM (“byte ordre mark” or header) on left side: two first bytes
    • See one byte for “!”
    • See two bytes each for “é”  and “É«
    • See three bytes for “用”

AutoHotkey Merchandise-White Stress ball

4) File Encoding Lab Cheat Sheet

  • ASCII and ANSI
  • Unicode 8-16-32 bits
  • Unicode with/without BOM
  • Unicode Little or Big Endian

5) Real life files

  • QAP Spanish translation
  • QAP Chinese translation
  • Other examples?

AutoHotkey Webinar- Parsing plain text with AutoHotkey

Video Hour 1: High-level overview:AutoHotkey Merchandise-White Stress ball

In the first hour of our AutoHotkey webinar we demonstrated Quick Access Popup from our guest speaker Jean Lalonde.  It is an amazingly customizable tool that allows you to have what you want at your fingertips!

Here is an abbreviated list of some of the great features:

  1. Create shortcuts to folders, programs, files, links, Snippets, special folders, etc.
  2. Easily create Hotkeys for above shortcuts
  3. Works with most Open/Save dialog boxes
  4. Integrated with most file managers (Windows Explorer, Directory Opus, Total Commander, etc.)
  5. The tool itself is highly customizable
  6. Have “shared” sections as well as individual ones
  7. Organize your menu into folders & sub-folders
  8. Customize Menu/GUI shortcut with an icon (that makes sense to you)
  9. Available in 10 languages

You can watch a short overview video here or get a deeper dive here.  Jean was kind enough to walk me through his tool in this video.

We then talked about what a “text file” is and frequent sources of them (Export from programs, sever logs, emails, etc).  We also briefly discussed what is File / Character Encoding & why are there frequent problems with it (this is a very deep topic which we could have spent days discussing!)

Also discussed  what a “flat file” is and how they are different than:

  • Multi-row files
  • Relational files
  • XML and JSON files

We then shared some of the frequently used built-in commands and functions for parsing text in AutoHotkey.

Two women smiling and looking at a computer screen together in a bright office, one pointing at the monitor.Then we got into various custom-built in libraries and functions

Then our guest speaker, Jean, took over walking us through some examples of common problems with AutoHotkey LoopParse command

The library is pretty cool and here are some of the main functions available:

  • CSV2Collection() – Transfer the content of a CSV file to a collection of objects.
  • Collection2CSV() – Transfer the selected fields from a collection of objects to a CSV file.
  • Collection2Fixed() – Transfer the selected fields from a collection of objects to a fixed-width file.
  • Collection2HTML() – Builds an HTML file based on a template where variables are replaced with content.
  • Collection2XML() – Builds an XML file from the content of the collection.
  • Collection2ListView() – Transfer the selected fields from a collection of objects to ListView.
  • ListView2Collection() – Transfer the selected lines of the selected columns of a ListView to a collection of objects.

ObjCSV Examples:  Here are the files Jean worked through.

  • SIMPLE CSV FILE – COMA DELIMITED (TheBeatles-coma.txt)
  • SIMPLE CSV FILE – TAB DELIMITED – ISSUE #1 (TheBeatles-tab.txt)
  • MULTI-LINE CSV FILE – COMA DELIMITED – ISSUE #2 (TheBeatles-Lyrics.txt)

Jean then gave us a review of how to use his ObjCSV library

  • Example using ObjCSV_CSV2Collection
  • SIMPLE CSV FILE – COMA DELIMITED (TheBeatles-coma.txt)
  • SIMPLE CSV FILE – TAB DELIMITED – ISSUE #1 SOLVED (TheBeatles-tab.txt)
  • MULTI-LINE CSV FILE – COMA DELIMITED – ISSUE #2 SOLVED (TheBeatles-Lyrics.txt)
  • PROGRESS BAR WITH LARGE CSV FILE – SEMI-COLON DELIMITED (LargeFile-semicolon.txt)
  • CUSTOM HEADER – SIMPLE CSV FILE – COMA DELIMITED (TheBeatles-coma.txt)
  • GUI Example using Collection2ListView
  • SAVING WITH DIFFERENT DELIMITER using ObjCSV_Collection2CSV
  • EXPORT HTML Example using ObjCSV_Collection2HTML

Wrapping up the first section we then saw a peak at CSVBuddy

CSVBuddy Examples

  • Reading large file with auto-detection
  • Editing field names, re-order or select fields
  • Saving options
  • Saving file with multi-line content converted to single-line (Excel-ready)
  • Reverting the single-line file to multi-line (for example, after manipulation in Excel)

Video Hour 2: Q&A:

In the second hour we discussed various issues working with text data as well as took a deeper look into Quick Access Popup.

Near the end of the time Capn Odin shared a Trie script he’s been working on.  You can check out the script on the forum

Clear AutoHotkey water bottle with green lid and band, showing impact resistance, BPA free, fade resistant, and durability claims

Extract email address from text with Regular Expression in AutoHotkey

Extract email addressA white stress ball with the green AutoHotkey logo being held in a hand, with a laptop keyboard blurred in the backgroundRegular Expressions are incredibly helpful for day-to-day tasks!  In this short video I demonstrate how easy it is to use a Regular Expression (RegEx) in AutoHotkey to extract email addresses from text.  The first half of the video shows how I use it to parse the haystack listed in the script.  The second half of the video I make a minor tweak and adapt it to work with any program that you can copy text to the clipboard.

Below is the code to extract emails with a Regular Expression that I demonstrate in the video

Extract email addresses via Regular Expression in AutoHotkey

A black AutoHotkey branded water bottle on a desk beside a laptop, with a person reading a newspaper in the background