From: Predicting user mental states in spoken dialogue systems
Description of the metrics in Table 3 | Simulated users | Recruited users | ||||||
---|---|---|---|---|---|---|---|---|
Baseline | Mental-state | Baseline | Mental-state | |||||
S1 | S2 | S1 | S2 | S1 | S2 | S1 | S2 | |
%success | 78 | 66 | 87 | 76 | 87 | 83 | 97 | 95 |
n CE | 0.76 | 0.71 | 0.89 | 0.84 | 0.85 | 0.80 | 0.91 | 0.88 |
n NCE | 0.21 | 0.24 | 0.09 | 0.11 | 0.18 | 0.20 | 0.09 | 0.08 |
%ECR | 79 | 75 | 91 | 88 | 82 | 80 | 92 | 91 |
avgturn/dial | 8.4 | 14.8 | 4.7 | 9.2 | 9.2 | 15.1 | 5.8 | 10.4 |
%diff | 76 | 88 | 67 | 84 | 77 | 93 | 76 | 91 |
#repMS | 7 | 3 | 9 | 7 | 5 | 2 | 8 | 4 |
#turnsMS | 2 | 9 | 2 | 7 | 2 | 9 | 2 | 7 |
#turnsSh | 2 | 7 | 2 | 7 | 2 | 7 | 2 | 7 |
#turnsLo | 14 | 20 | 12 | 18 | 12 | 17 | 9 | 15 |