User manual SPSS DATA VALIDATION 14.0

[. . . ] SPSS Data Validation 14. 0 TM For more information about SPSS® software products, please visit our Web site at http://www. spss. com or contact SPSS Inc. 233 South Wacker Drive, 11th Floor Chicago, IL 60606-6412 Tel: (312) 651-3000 Fax: (312) 651-3668 SPSS is a registered trademark and the other product names are the trademarks of SPSS Inc. No material describing such software may be produced or distributed without the written permission of the owners of the trademark and license rights in the software and the copyrights in the published materials. Use, duplication, or disclosure by the Government is subject to restrictions as set forth in subdivision (c) (1) (ii) of The Rights in Technical Data and Computer Software clause at 52. 227-7013. [. . . ] The procedure is designed to quickly detect unusual cases for data-auditing purposes in the exploratory data analysis step, prior to any inferential data analysis. This algorithm is designed for generic anomaly detection; that is, the definition of an anomalous case is not specific to any particular application, such as detection of unusual payment patterns in the healthcare industry or detection of money laundering in the finance industry, in which the definition of an anomaly can be well-defined. Example. A data analyst hired to build predictive models for stroke treatment outcomes is concerned about data quality because such models can be sensitive to unusual observations. Some of these outlying observations represent truly unique cases and are thus unsuitable for prediction, while other observations are caused by data entry errors in which the values are technically "correct" and thus cannot be caught by data validation procedures. The Identify Unusual Cases procedure finds and reports these outliers so that the analyst can decide how to handle them. Statistics. The procedure produces peer groups, peer group norms for continuous and categorical variables, anomaly indices based on deviations from peer group norms, and variable impact values for variables that most contribute to a case being considered unusual. Data Considerations Data. Each row represents a distinct observation, and each column represents a distinct variable upon which the peer groups are based. A case identification variable can be available in the data file for marking output, but it will not be used in the analysis. The SPSS weight variable, if specified, is ignored. 21 22 Chapter 4 The detection model can be applied to a new test data file. The elements of the test data must be the same as the elements of the training data. And, depending on the algorithm settings, the missing value handling that is used to create the model may be applied to the test data file prior to scoring. Case order. To verify the stability of a given solution, you may want to obtain several different solutions with cases sorted in different random orders. In situations with extremely large file sizes, multiple runs can be performed, with a sample of cases sorted in different random orders. Assumptions. The algorithm assumes that all variables are nonconstant and independent and assumes that no case has missing values for all of the input variables. Each continuous variable is assumed to have a normal (Gaussian) distribution, and each categorical variable is assumed to have a multinomial distribution. Empirical internal testing indicates that the procedure is fairly robust to violations of both the assumption of independence and the distributional assumptions, but be aware of how well these assumptions are met. To Identify Unusual Cases E From the menus choose: Data Identify Unusual Cases. . . 23 Identify Unusual Cases Figure 4-1 Identify Unusual Cases: Variables tab E Select at least one analysis variable. E Optionally, choose a case ID variable to use in labeling output. 24 Chapter 4 Identify Unusual Cases Output Figure 4-2 Identify Unusual Cases: Output tab List of unusual cases and reasons why they are considered unusual. This option produces three tables: The anomaly case index list displays cases that are identified as unusual and displays their corresponding anomaly index values. The anomaly case peer ID list displays unusual cases and information concerning their corresponding peer groups. The anomaly reason list displays the case number, the reason variable, the variable impact value, the value of the variable, and the norm of the variable for each reason. Moreover, the IDs of the cases are displayed if the case identifier variable is specified on the Variables tab. 25 Identify Unusual Cases Summaries. [. . . ] In contrast, Total treatment and rehabilitation costs in thousands and Missing Proportion each provide some insight into peer group membership. Peer group 1 has the highest average cost and the fewest missing values. This organization suggests that peer group 2 is composed of patients who were dead on arrival, thus incurring very little cost and causing all of the treatment and rehabilitation variables to be missing. Peer group 3 likely contains many patients who died during treatment, thus incurring the treatment costs but not the rehabilitation costs and causing the rehabilitation variables to be missing. [. . . ]