Forum de leno - memo & co

depôt d'url utiles et variées
 
AccueilAccueil  FAQFAQ  RechercherRechercher  S'enregistrerS'enregistrer  MembresMembres  GroupesGroupes  Connexion  

Partagez | 
 

 benchmarks hboa

Voir le sujet précédent Voir le sujet suivant Aller en bas 
AuteurMessage
leno



Nombre de messages : 1051
Date d'inscription : 19/07/2006

MessageSujet: benchmarks hboa   Ven 22 Aoû - 8:55

75% de zeros
1200 population
0.015

1110000010010100000010100000011010000100000011000000000001010000000000000000100110000000000000000000
sorting used : naive bayes
0 0.333333
sorting used : naive bayes
0.333333 0.666667
sorting used : naive bayes
0.666667 1
truePositive : 3242
trueNegative : 3175
falsePositive : 1825
falseNegative : 1758

15/19
64.17%
Revenir en haut Aller en bas
Voir le profil de l'utilisateur
leno



Nombre de messages : 1051
Date d'inscription : 19/07/2006

MessageSujet: Re: benchmarks hboa   Ven 22 Aoû - 9:09

full vector
DecisionTable
3folds
about 1mn

=== Run information ===

Scheme: weka.classifiers.rules.DecisionTable -X 1 -S "weka.attributeSelection.BestFirst -D 1 -N 5"
Relation: cpu
Instances: 10000
Attributes: 101
[list of attributes omitted]
Test mode: 3-fold cross-validation

=== Classifier model (full training set) ===

Decision Table:

Number of training instances: 10000
Number of Rules : 238
Non matches covered by Majority class.
Best first.
Start set: no attributes
Search direction: forward
Stale search after 5 node expansions
Total number of subsets evaluated: 961
Merit of best subset found: 69.76
Evaluation (for feature selection): CV (leave one out)
Feature set: 30,36,45,58,76,101

Time taken to build model: 27.75 seconds

=== Stratified cross-validation ===
=== Summary ===

Correctly Classified Instances 6872 68.72 %
Incorrectly Classified Instances 3128 31.28 %
Kappa statistic 0.3744
Mean absolute error 0.4019
Root mean squared error 0.4508
Relative absolute error 80.3745 %
Root relative squared error 90.1597 %
Total Number of Instances 10000

=== Detailed Accuracy By Class ===

TP Rate FP Rate Precision Recall F-Measure ROC Area Class
0.71 0.336 0.679 0.71 0.694 0.749 c1
0.664 0.29 0.696 0.664 0.68 0.749 c2

=== Confusion Matrix ===

a b <-- classified as
3551 1449 | a = c1
1679 3321 | b = c2

68.72%















ideal vector
DecisionTable
3folds
about 15s

=== Run information ===

Scheme: weka.classifiers.rules.DecisionTable -X 1 -S "weka.attributeSelection.BestFirst -D 1 -N 5"
Relation: cpu
Instances: 10000
Attributes: 26
0
1
2
8
13
20
27
29
30
32
35
37
38
40
42
44
46
50
55
57
59
75
79
80
88
play
Test mode: 3-fold cross-validation

=== Classifier model (full training set) ===

Decision Table:

Number of training instances: 10000
Number of Rules : 238
Non matches covered by Majority class.
Best first.
Start set: no attributes
Search direction: forward
Stale search after 5 node expansions
Total number of subsets evaluated: 209
Merit of best subset found: 69.76
Evaluation (for feature selection): CV (leave one out)
Feature set: 8,11,16,20,22,26

Time taken to build model: 5.59 seconds

=== Stratified cross-validation ===
=== Summary ===

Correctly Classified Instances 6872 68.72 %
Incorrectly Classified Instances 3128 31.28 %
Kappa statistic 0.3744
Mean absolute error 0.4019
Root mean squared error 0.4508
Relative absolute error 80.3745 %
Root relative squared error 90.1597 %
Total Number of Instances 10000

=== Detailed Accuracy By Class ===

TP Rate FP Rate Precision Recall F-Measure ROC Area Class
0.71 0.336 0.679 0.71 0.694 0.749 c1
0.664 0.29 0.696 0.664 0.68 0.749 c2

=== Confusion Matrix ===

a b <-- classified as
3551 1449 | a = c1
1679 3321 | b = c2

68.72%


















18/23 vector
DecisionTable
3folds
about 15s

=== Run information ===

Scheme: weka.classifiers.rules.DecisionTable -X 1 -S "weka.attributeSelection.BestFirst -D 1 -N 5"
Relation: cpu
Instances: 10000
Attributes: 24
0
1
2
8
11
13
16
20
29
37
42
44
46
55
57
59
75
76
79
80
88
96
97
play
Test mode: 3-fold cross-validation

=== Classifier model (full training set) ===

Decision Table:

Number of training instances: 10000
Number of Rules : 353
Non matches covered by Majority class.
Best first.
Start set: no attributes
Search direction: forward
Stale search after 5 node expansions
Total number of subsets evaluated: 190
Merit of best subset found: 69.67
Evaluation (for feature selection): CV (leave one out)
Feature set: 2,9,12,15,17,24

Time taken to build model: 5.11 seconds

=== Stratified cross-validation ===
=== Summary ===

Correctly Classified Instances 6766 67.66 %
Incorrectly Classified Instances 3234 32.34 %
Kappa statistic 0.3532
Mean absolute error 0.4063
Root mean squared error 0.4538
Relative absolute error 81.2526 %
Root relative squared error 90.7529 %
Total Number of Instances 10000

=== Detailed Accuracy By Class ===

TP Rate FP Rate Precision Recall F-Measure ROC Area Class
0.689 0.336 0.672 0.689 0.681 0.742 c1
0.664 0.311 0.681 0.664 0.673 0.742 c2

=== Confusion Matrix ===

a b <-- classified as
3445 1555 | a = c1
1679 3321 | b = c2

67.66%
Revenir en haut Aller en bas
Voir le profil de l'utilisateur
leno



Nombre de messages : 1051
Date d'inscription : 19/07/2006

MessageSujet: Re: benchmarks hboa   Ven 22 Aoû - 9:26

the goal of feature selection is 3fold

-improve prediction performance
- provide more cost-effective prediction
- better understanding of the underlying process that generated the data

a feature selection method generates candidates from the feature space and asses them based on some evaluation criterion to find the best feature subset
Revenir en haut Aller en bas
Voir le profil de l'utilisateur
Contenu sponsorisé




MessageSujet: Re: benchmarks hboa   

Revenir en haut Aller en bas
 
benchmarks hboa
Voir le sujet précédent Voir le sujet suivant Revenir en haut 
Page 1 sur 1

Permission de ce forum:Vous ne pouvez pas répondre aux sujets dans ce forum
Forum de leno - memo & co :: Forum de leno - memo & co-
Sauter vers: