AutoHotkey webinar: Deep-Dive into File Encoding

In this AutoHotkey Webinar took a deep-dive into File Encoding by our guest speaker Jean Lalonde (author of Quick Access Popup)

Video Hour 1:  High Level:

• Why File encoding matters
• Pros/Cons of each type of encoding
• Tools like the File Encoding Lab to help determine a file’s encoding
• How to set File Encoding in AutoHotkey

Video Hour 2: Coding and Q&A

• Other mentions in Hour 2:
  • CodeQuickTester helps quickly test code w/o saving
  • New Hotstrings command
  • Neural Networks (topic for March webinar)

Script Highlight: BarChart by Learning One

(Here is a link to BarChart scripts demonstrated during the webinar)

1) File Enconding in AutoHotkey

(Here is a link to all of Jean Lalonde’s files)

• Edit and run the AHK script “FileRead.ahk”
• Load “Demo-UTF-16.txt” with default encoding
• Load “Demo-UTF-16-No_BOM.txt” with default encoding  -> problem!
• Load “Demo-UTF-16-No_BOM.txt” with UTF-16-RAW encoding  -> OK!
• Try the other file enconding available in AHK

2) File Encoding in DOS (code pages)

• Run the batch file “Type Box.bat” in DOS console (under “Tutorial“ folder)
• See this ASCII/ANSI file displayed with code pages 1252 (default ASCI/ANSI) and 437

3) Load the File Encoding Lab

• Run the AHK script “File Encoding Lab.ahk”
• Loaded bu default “ASCII.txt” (detected CP1252, 7-bit chars)
• File Encoding Lab tour
  • Binary display on the left side / Normal AHK display on the right
  • Click on the file name to see with Notepad
  • Encoded detected and use the dropdown list to reload with another encoding

• Load “ANSI.txt” (detected CP1252, 8-bit chars)
• Load “Box-CP437.txt” (detected CP1252, 8-bit chars)
  • Which encoding will display a box?

• Load “UTF-8.txt” (detected UTF-8)
  • see BOM (“byte ordre mark” or header) on left side: two first bytes
  • See one byte for “!”
  • See two bytes each for “é”  and “É«
  • See three bytes for “用”

4) File Encoding Lab Cheat Sheet

• ASCII and ANSI
• Unicode 8-16-32 bits
• Unicode with/without BOM
• Unicode Little or Big Endian

5) Real life files

• LinkedIn Connections Export UTF-8 No BOM
  • open “Connection.csv” in MS Excel
  • convert it to ANSI or UTF-16 with CSV Buddy (http://code.jeanlalonde.ca/csvbuddy/)

• QAP Spanish translation
• QAP Chinese translation
• Other examples?