Merging Files with similar structure but removing duplicate headers

I often get text files of similar type (Tab delimited, CSV, etc.) but they all have a header row.  A super easy way to merge the data together is to

  1. Created a folder that only has the files you want to merge
  2. create a command prompt in the directory
  3. use the following dos command to merge the files (adjusting the extension to your purpose:  copy *.csv  merged.csv

This will merge all of the files ending in .csv into the merged.csv file however it keeps all the header rows. :(   This annoyed me so I wrote a short program to use AutoHotkey and only keep the first header row.

Continue reading

Updated iWB2 Learner tool to grab ClassName & Highlight Frames

ClassName & Highlight FramesI use ClassName a lot when web scraping.  So much so I updated the iWB2 Learner tool to extract it from the outerHTML and have it’s own edit field (like Name and ID).

I also changed the ListView that shows Frames to be RED.  I can’t tell you how much time I wasted early-on when dealing with Frames.  Hopefully this will help people realize they need to take a different approach when Frames are present! Continue reading