SPSS macro to automatically remove duplicates and drop variable

SPSS macro for removing duplicates

SPSS has a built in GUI function for removing duplicates however it is lacking some of the key options that I like to use (perform case insensitive dedupe, drop duplicates, drop the flag, etc). I built my own macro which is very easy to use and has all the options I typically need.

The following video demonstrates them in use.

Here are the SPSS macros

*///////////////.
DEFINE !DDN (DupVar !TOKENS (1))
SORT CASES BY !DupVar (A) .
MATCH FILES /FILE = *  /BY !DupVar /FIRST = First.
var label first "Unique and Duplicates".
value label first 0'Duplicates' 1'Unique'.
Freq First /FORMAT=DVALUE.
select if First=1.
match files file= * / drop First.
exe.
!ENDDEFINE.
*///////////////.
/* !DDN DupVar=Number.

*////SPSS macro to remove duplicates on String variable///////////.
DEFINE !DDA (DupVar !TOKENS (1))
String duh (A1000).
SORT CASES BY !DupVar (A) .
compute duh=Lowcase(!DupVar).
MATCH FILES /FILE = *  /BY Duh /FIRST = First.
var label first "Unique and Duplicates".
value label first 0'Duplicates' 1'Unique'.
Freq First /FORMAT=DVALUE.
select if First=1.
match files file= * / drop First duh.
exe.
!ENDDEFINE.
*///////////////.
/* !DDA DupVar=email.

*///////////////.
DEFINE !DDM  (Var1 !TOKENS (1)   / Var2 !Tokens (1) )
String duh1 duh2 (A1000).
Compute duh1=Lowcase(!Var1).
Compute duh2=Lowcase(!Var2).
SORT CASES BY Duh1 (A) Duh2 (A).
MATCH FILES /FILE = * /BY Duh1  Duh2 /FIRST =First.
var label first "Unique and Duplicates".
value label first 0'Duplicates' 1'Unique'.
Freq First /FORMAT=DVALUE.
select if First=1.
match files file= * / drop First Duh1 Duh2.
exe.
EXECUTE.
!ENDDEFINE.
*///////////////.
/* !DDM Var1=Corporate  Var2=gend.


*/////SPSS macro to remove duplicates on two string variables//////////.
DEFINE !DDA2 (DupVar !TOKENS (1))
SORT CASES BY !DupVar.
Compute Unique=1.
if Upcase(!DupVar)=Upcase(Lag(!DupVar)) Unique=0.
Freq Unique.
Select if Unique=1.
Match files file=* / DROP Unique.
exe.
!ENDDEFINE.
*///////////////.
/*!DDA2  DupVar=email.

SPSS Syntax writer / Recoder using an Excel template

SPSS syntax

Writing (and re-writing) SPSS syntax can be mundane and time consuming. Sometimes my data set has hundreds of variables which would take me quite some time to rename and relabel. I created an Excel template which generates the necessary syntax and greatly helped automate the process.

The Excel file is available for download from here and the video below helps you see how easy it can be to use. There is also a concatenation tab which allows you to add specific words as a prefix for your variables.

Setting output for SPSS Variable names, Labels, or both

Typically I want to have the SPSS variable labels show up in my SPSS output however there are times when having the variable names is beneficial (especially for writing more syntax). Other times (rare) I actually want to have both. The syntax for changing is straight forward

 
*****Show Names*********.
SET OVars Labels ONumbers Labels TVars Names TNumbers Labels.

*****Show Labels********.
SET OVars Labels ONumbers Labels TVars Labels TNumbers Labels.

*****Show Both**********.
SET OVars Labels ONumbers Labels TVars Both TNumbers Labels.

video demonstrating using the syntax on the SPSS Variable Names

Demystifying the Ternary Operator in AutoHotkey: Short-hand If-Else logic

AutoHotkey has pretty straight-forward If-Then-Else logic however it can take quite a few lines of code to do some simple evaluations. Ternary Operators are a great way to write short / concise code in one line! 🙂

The basic form is that you first evaluate a condition. If the value resolves to being true, the first parameter is returned. If it is False, the second parameter is returned.

(Condition)?(True) : (False)