The aim of the present paper is to collect evidence about validity of the cognitive structure proposed to solve the items of an arithmetic test using an Item Response Theory based cognitive diagnosis model called Least Squares Distance Model (LSDM). The test was applied to a sample of 382 students of 7th grade from five public high schools in Bogotá-Colombia. With this data the objective was addressed in three ways: first, the analysis of two statistical validity indices; second by a cross-validation of the LSDM results on attribute difficulties using other cognitive model (LLTM), and third, by comparing the LSDM results with the observed scores on individual attributes. The logical behavior of the probability curves for the five attributes under study provides important evidence for heir overall validity. Additional specific evidences of validity are also presented in the results.