Home » Forums » decision trees & data mining forum

Test set performance

Submitted by cap on Mon, 2007-04-23 15:54.
What test set performance % would you say is the maximum one should observe when dealing with a marketing/response model. With the model I'm working on right now I am getting 90% -93% test performance on all my runs. I know in my logit experience anytime I got 85% on test set performance it was questionable as to the reliability of the model. Is this the same with classification modeling? Can such high percentage performance be considered valid?
‹ How to explain decision Tree. Missing values ›
»

re: Test set performance

Submitted by michaelN on Mon, 2007-04-30 14:54.
It's not totally unusual to see such high accuracy of a model on test data if your test data is similar to your training data. You probably want to try one of the validation methods to avoid this although it could just be that the data exhibits little noise and variation and can be modelled easily.
»