Model performance
News: Trankit v1.0.0 is out. The new version has pretrained pipelines using XLM-Roberta Large which is much better than our previous version with XLM-Roberta Base. Checkout the performance comparison below on this page. Trankit v1.0.0 also provides a brand new command-line interface that helps users who are not familiar with Python can use Trankit more easily. Finally, the new version has a brand new Auto Mode in which a language detector is used to activate the language-specific models for processing the input, avoiding switching back and forth between languages in a multilingual pipeline.
Universal Dependencies v2.5
The following table shows the performance comparison between Trankit large (using XLM-Roberta large) and base (using XLM-Roberta base), spaCy v2.3, UDPipe v1.2, and Stanza v1.1.1 on the test treebanks of the 5 major languages on the Universal Dependencies v2.5 corpora. Performance for each model are F1 scores obtained by using the official evaluation script of the CoNLL 2018 Shared Task. The meanings of the columns in the following tables are provided by the evaluation script as follows:
Tokens: how well do the gold tokens match system tokens.
Sents.: how well do the gold sentences match system sentences.
Words: how well can the gold words be aligned to system words.
UPOS: using aligned words, how well does UPOS match.
XPOS: using aligned words, how well does XPOS match.
UFeats: using aligned words, how well does universal FEATS match.
AllTags: using aligned words, how well does UPOS+XPOS+FEATS match.
Lemmas: using aligned words, how well does LEMMA match.
UAS: using aligned words, how well does HEAD match.
LAS: using aligned words, how well does HEAD+DEPREL(ignoring subtypes) match.
CLAS: using aligned words with content DEPREL, how well does HEAD+DEPREL(ignoring subtypes) match.
MLAS: using aligned words with content DEPREL, how well does HEAD+DEPREL(ignoring subtypes)+UPOS+UFEATS+FunctionalChildren(DEPREL+UPOS+UFEATS) match.
BLEX: using aligned words with content DEPREL, how well does HEAD+DEPREL(ignoring subtypes)+LEMMAS match.
Treebank |
System |
Tokens |
Sents. |
Words |
UPOS |
XPOS |
UFeats |
Lemmas |
UAS |
LAS |
Arabic-PADT |
Trankit[large] |
99.95 |
96.79 |
99.39 |
96.8 |
94.93 |
94.99 |
94.92 |
89.74 |
85.98 |
Trankit[base] |
99.93 |
96.59 |
99.22 |
96.31 |
94.08 |
94.28 |
94.65 |
88.39 |
84.68 |
|
Stanza |
99.98 |
80.43 |
97.88 |
94.89 |
91.75 |
91.86 |
93.27 |
83.27 |
79.33 |
|
UDPipe |
99.98 |
82.09 |
94.58 |
90.36 |
84.00 |
84.16 |
88.46 |
72.67 |
68.14 |
|
Chinese-GSD |
Trankit[large] |
97.75 |
99.4 |
97.75 |
95.13 |
94.94 |
97.28 |
97.75 |
87.38 |
84.82 |
Trankit[base] |
97.01 |
99.7 |
97.01 |
94.21 |
94.02 |
96.59 |
97.01 |
85.19 |
82.54 |
|
Stanza |
92.83 |
98.80 |
92.83 |
89.12 |
88.93 |
92.11 |
92.83 |
72.88 |
69.82 |
|
UDPipe |
90.27 |
99.10 |
90.27 |
84.13 |
84.04 |
89.05 |
90.26 |
61.60 |
57.81 |
|
English-EWT |
Trankit[large] |
98.67 |
90.49 |
98.67 |
96.65 |
96.47 |
96.75 |
97.25 |
91.29 |
89.4 |
Trankit[base] |
98.48 |
88.35 |
98.48 |
95.95 |
95.71 |
96.26 |
96.84 |
90.14 |
87.96 |
|
Stanza |
99.01 |
81.13 |
99.01 |
95.40 |
95.12 |
96.11 |
97.21 |
86.22 |
83.59 |
|
UDPipe |
98.90 |
77.40 |
98.90 |
93.26 |
92.75 |
94.23 |
95.45 |
80.22 |
77.03 |
|
spaCy |
97.30 |
61.19 |
97.30 |
86.72 |
90.83 |
87.05 |
||||
French-GSD |
Trankit[large] |
99.77 |
98.8 |
99.72 |
97.89 |
97.34 |
97.9 |
94.97 |
93.37 |
|
Trankit[base] |
99.7 |
96.63 |
99.66 |
97.85 |
97.16 |
97.80 |
94.00 |
92.34 |
||
Stanza |
99.68 |
94.92 |
99.48 |
97.30 |
96.72 |
97.64 |
91.38 |
89.05 |
||
UDPipe |
99.68 |
93.59 |
98.81 |
95.85 |
95.55 |
96.61 |
87.14 |
84.26 |
||
spaCy |
98.34 |
77.30 |
94.15 |
86.82 |
87.29 |
67.46 |
60.60 |
|||
Spanish-Ancora |
Trankit[large] |
99.91 |
99.39 |
99.91 |
99.06 |
99 |
98.85 |
99.15 |
94.77 |
93.29 |
Trankit[base] |
99.94 |
99.13 |
99.93 |
99.02 |
98.94 |
98.80 |
99.17 |
94.11 |
92.41 |
|
Stanza |
99.98 |
99.07 |
99.98 |
98.78 |
98.67 |
98.59 |
99.19 |
92.21 |
90.01 |
|
UDPipe |
99.97 |
98.32 |
99.95 |
98.32 |
98.13 |
98.13 |
98.48 |
88.22 |
85.10 |
|
spaCy |
99.47 |
97.59 |
98.95 |
94.04 |
79.63 |
86.63 |
84.13 |
As can be seen from the table, both the large and base version of Trankit outperform other toolkits over different tasks (e.g., POS and morphological tagging) in which the improvement boost is substantial and significant for sentence segmentation and dependency parsing. For example, English enjoys a 9.36% improvement for sentence segmentation, a 5.07% and 5.81% improvement for UAS and LAS in dependency parsing. For Arabic, Transit has a remarkable improvement of 16.36% for sentence segmentation while Chinese observes 14.50% and 15.0% improvement of UAS and LAS for dependency parsing.
Next, we show the detailed performance comparison of Trankit large (using XLM-Roberta large), Trankit base (using XLM-Roberta base), and Stanza v1.1.1 on 90 Universal Dependencies v2.5 treebanks. Over all 90 treebanks, both versions of Trankit outperform the previous state-of-the-art framework Stanza in most of the tasks, particularly for sentence segmentation (+4.28%), POS tagging (+2.00% for UPOS and +2.14% for XPOS), morphological tagging (+2.18%), and dependency parsing (+5.61% for UAS and +6.81% for LAS) while maintaining the competitive performance on tokenization, multi-word token expansion, and lemmatization.
Tokens |
Sentences |
Words |
UPOS |
XPOS |
UFeats |
AllTags |
Lemmas |
UAS |
LAS |
CLAS |
MLAS |
BLEX |
||
Overall performance |
Trankit[large] |
99.32 |
92.86 |
99.15 |
96.22 |
94.64 |
93.93 |
91.19 |
94.47 |
88.67 |
85.49 |
82.97 |
76.17 |
78.45 |
Trankit[base] |
99.23 |
91.82 |
99.02 |
95.65 |
94.05 |
93.21 |
90.29 |
94.27 |
87.06 |
83.69 |
80.88 |
73.57 |
76.53 |
|
Stanza |
99.26 |
88.58 |
98.90 |
94.21 |
92.50 |
91.75 |
88.62 |
94.15 |
83.06 |
78.68 |
74.65 |
67.83 |
71.28 |
|
Trankit[large] - Trankit[base] |
0.09 |
1.04 |
0.13 |
0.56 |
0.59 |
0.72 |
0.91 |
0.20 |
1.61 |
1.80 |
2.10 |
2.60 |
1.92 |
|
Trankit[large] - Stanza |
0.07 |
4.28 |
0.26 |
2.00 |
2.14 |
2.18 |
2.57 |
0.32 |
5.61 |
6.81 |
8.32 |
8.34 |
7.17 |
|
Afrikaans-AfriBooms |
Trankit[large] |
99.84 |
100.00 |
99.84 |
98.87 |
96.63 |
98.68 |
96.54 |
97.47 |
91.61 |
89.35 |
85.30 |
82.64 |
81.65 |
Trankit[base] |
99.72 |
100.00 |
99.72 |
98.55 |
96.10 |
98.26 |
95.89 |
97.39 |
91.03 |
88.79 |
84.46 |
81.31 |
80.91 |
|
Stanza |
99.75 |
99.65 |
99.75 |
97.56 |
94.27 |
97.03 |
94.24 |
97.48 |
87.51 |
84.45 |
78.58 |
74.70 |
75.39 |
|
Ancient_Greek-Perseus |
Trankit[large] |
99.71 |
98.70 |
99.71 |
93.97 |
87.25 |
91.66 |
86.88 |
88.52 |
83.48 |
78.56 |
73.79 |
60.72 |
61.97 |
Trankit[base] |
99.66 |
98.70 |
99.66 |
92.72 |
84.89 |
89.95 |
84.30 |
88.18 |
80.95 |
75.57 |
70.26 |
55.75 |
58.84 |
|
Stanza |
99.98 |
98.85 |
99.98 |
92.54 |
85.22 |
91.06 |
84.98 |
88.26 |
78.75 |
73.35 |
67.88 |
54.22 |
57.54 |
|
Ancient_Greek-PROIEL |
Trankit[large] |
99.91 |
67.60 |
99.91 |
97.86 |
97.93 |
93.03 |
92.02 |
97.50 |
85.63 |
82.31 |
78.16 |
68.27 |
75.76 |
Trankit[base] |
99.83 |
58.62 |
99.83 |
97.30 |
97.37 |
91.59 |
90.33 |
97.31 |
83.21 |
79.68 |
74.96 |
64.13 |
72.80 |
|
Stanza |
100.00 |
51.65 |
100.00 |
97.38 |
97.75 |
92.09 |
90.96 |
97.42 |
80.34 |
76.33 |
71.37 |
61.23 |
69.23 |
|
Arabic-PADT |
Trankit[large] |
99.95 |
96.79 |
99.39 |
96.80 |
94.93 |
94.99 |
94.61 |
94.92 |
89.74 |
85.98 |
83.92 |
78.47 |
79.96 |
Trankit[base] |
99.93 |
96.59 |
99.22 |
96.31 |
94.08 |
94.28 |
93.69 |
94.65 |
88.39 |
84.68 |
82.35 |
76.46 |
78.46 |
|
Stanza |
99.98 |
80.43 |
97.88 |
94.89 |
91.75 |
91.86 |
91.51 |
93.27 |
83.27 |
79.33 |
76.24 |
70.58 |
72.79 |
|
Armenian-ArmTDP |
Trankit[large] |
98.52 |
99.46 |
97.26 |
95.03 |
97.26 |
90.42 |
89.56 |
93.45 |
86.47 |
83.01 |
80.91 |
72.45 |
77.07 |
Trankit[base] |
98.41 |
98.92 |
97.23 |
94.36 |
97.23 |
88.89 |
87.84 |
93.28 |
84.22 |
80.14 |
77.47 |
67.92 |
73.94 |
|
Stanza |
98.96 |
95.22 |
96.58 |
92.49 |
96.58 |
88.19 |
86.94 |
92.27 |
78.18 |
72.46 |
68.50 |
60.39 |
65.88 |
|
Basque-BDT |
Trankit[large] |
99.89 |
100.00 |
99.89 |
96.95 |
99.89 |
93.96 |
92.30 |
96.69 |
90.22 |
87.47 |
86.54 |
78.10 |
83.21 |
Trankit[base] |
99.81 |
99.92 |
99.81 |
96.23 |
99.81 |
92.35 |
90.40 |
96.46 |
88.24 |
85.25 |
84.29 |
74.20 |
80.92 |
|
Stanza |
100.00 |
100.00 |
100.00 |
96.23 |
100.00 |
93.09 |
91.34 |
96.52 |
86.19 |
82.76 |
81.29 |
73.56 |
78.26 |
|
Belarusian-HSE |
Trankit[large] |
99.81 |
82.28 |
99.81 |
96.46 |
31.77 |
82.82 |
27.96 |
80.37 |
77.48 |
73.36 |
70.18 |
51.71 |
52.39 |
Trankit[base] |
99.53 |
84.68 |
99.53 |
91.58 |
36.18 |
78.47 |
27.19 |
79.48 |
69.40 |
65.02 |
64.51 |
46.24 |
48.41 |
|
Stanza |
99.38 |
78.24 |
99.38 |
91.92 |
31.34 |
77.73 |
26.31 |
79.48 |
69.28 |
63.88 |
58.49 |
41.88 |
44.05 |
|
Bulgarian-BTB |
Trankit[large] |
99.78 |
98.79 |
99.78 |
99.29 |
97.75 |
98.30 |
97.29 |
97.37 |
96.30 |
94.21 |
92.19 |
89.64 |
88.79 |
Trankit[base] |
99.84 |
98.25 |
99.84 |
99.18 |
97.42 |
98.10 |
96.91 |
97.36 |
95.81 |
93.47 |
91.23 |
88.28 |
87.80 |
|
Stanza |
99.93 |
97.27 |
99.93 |
98.68 |
96.35 |
97.59 |
95.75 |
97.29 |
93.37 |
90.21 |
86.84 |
83.71 |
83.62 |
|
Catalan-AnCora |
Trankit[large] |
99.94 |
99.76 |
99.94 |
99.11 |
99.07 |
98.66 |
98.24 |
98.49 |
95.15 |
93.83 |
91.41 |
89.31 |
90.27 |
Trankit[base] |
99.94 |
100.00 |
99.93 |
99.02 |
98.97 |
98.57 |
98.09 |
98.46 |
94.61 |
93.01 |
90.09 |
87.89 |
88.99 |
|
Stanza |
99.99 |
99.84 |
99.98 |
98.75 |
98.66 |
98.29 |
97.74 |
98.47 |
92.84 |
90.56 |
86.25 |
84.07 |
85.31 |
|
Chinese-GSD |
Trankit[large] |
97.75 |
99.40 |
97.75 |
95.13 |
94.94 |
97.28 |
94.45 |
97.75 |
87.38 |
84.82 |
83.44 |
79.82 |
83.44 |
Trankit[base] |
97.01 |
99.70 |
97.01 |
94.21 |
94.02 |
96.59 |
93.56 |
97.01 |
85.19 |
82.54 |
80.91 |
77.42 |
80.91 |
|
Stanza |
92.83 |
98.80 |
92.83 |
89.12 |
88.93 |
92.11 |
88.18 |
92.83 |
72.88 |
69.82 |
66.81 |
63.26 |
66.81 |
|
Classical_Chinese-Kyoto |
Trankit[large] |
99.70 |
70.58 |
99.70 |
92.89 |
91.90 |
94.43 |
89.98 |
99.67 |
82.03 |
76.82 |
75.60 |
73.16 |
75.60 |
Trankit[base] |
99.63 |
61.82 |
99.63 |
92.07 |
91.03 |
93.88 |
88.92 |
99.60 |
78.79 |
73.23 |
72.18 |
69.27 |
72.18 |
|
Stanza |
99.47 |
46.95 |
99.47 |
90.25 |
89.64 |
92.68 |
87.34 |
99.45 |
71.81 |
66.08 |
64.54 |
62.61 |
64.54 |
|
Croatian-SET |
Trankit[large] |
99.93 |
99.08 |
99.93 |
98.58 |
96.55 |
96.85 |
95.99 |
96.71 |
93.86 |
89.74 |
87.77 |
82.19 |
83.56 |
Trankit[base] |
99.92 |
99.16 |
99.92 |
98.38 |
96.08 |
96.52 |
95.44 |
96.60 |
93.34 |
89.36 |
87.16 |
81.12 |
82.91 |
|
Stanza |
99.96 |
98.15 |
99.96 |
97.88 |
94.86 |
95.32 |
94.22 |
96.67 |
90.27 |
85.56 |
82.43 |
76.37 |
78.78 |
|
Czech-CAC |
Trankit[large] |
99.99 |
100.00 |
99.98 |
99.64 |
98.17 |
98.01 |
97.65 |
98.30 |
95.48 |
93.97 |
93.06 |
90.13 |
90.92 |
Trankit[base] |
99.96 |
100.00 |
99.95 |
99.42 |
97.40 |
97.12 |
96.68 |
98.13 |
94.97 |
93.27 |
92.08 |
88.13 |
89.85 |
|
Stanza |
99.99 |
100.00 |
99.97 |
98.76 |
94.79 |
93.52 |
92.65 |
98.00 |
91.70 |
89.19 |
86.84 |
80.14 |
84.89 |
|
Czech-CLTT |
Trankit[large] |
99.89 |
98.89 |
99.85 |
99.17 |
93.80 |
94.02 |
93.53 |
97.81 |
92.53 |
90.76 |
88.90 |
81.07 |
86.35 |
Trankit[base] |
99.82 |
100.00 |
99.76 |
98.93 |
93.36 |
93.68 |
93.21 |
97.66 |
90.20 |
88.01 |
85.13 |
77.66 |
82.80 |
|
Stanza |
99.93 |
100.00 |
99.84 |
98.92 |
91.89 |
91.97 |
91.28 |
97.48 |
86.67 |
83.38 |
79.35 |
70.70 |
77.56 |
|
Czech-FicTree |
Trankit[large] |
99.98 |
99.50 |
99.98 |
99.11 |
97.26 |
97.99 |
97.06 |
98.64 |
96.50 |
94.85 |
93.44 |
89.79 |
91.46 |
Trankit[base] |
99.97 |
99.38 |
99.97 |
98.94 |
96.47 |
97.09 |
96.12 |
98.61 |
95.85 |
93.86 |
92.10 |
87.13 |
90.16 |
|
Stanza |
99.97 |
98.60 |
99.96 |
98.31 |
95.23 |
96.01 |
94.58 |
98.43 |
92.69 |
89.81 |
87.30 |
81.94 |
85.42 |
|
Czech-PDT |
Trankit[large] |
99.95 |
97.87 |
99.95 |
99.32 |
98.19 |
98.19 |
97.77 |
98.54 |
95.24 |
93.65 |
92.79 |
90.18 |
91.01 |
Trankit[base] |
99.94 |
97.85 |
99.94 |
99.23 |
97.81 |
97.77 |
97.34 |
98.49 |
94.81 |
93.18 |
92.09 |
89.11 |
90.33 |
|
Stanza |
99.97 |
94.14 |
99.97 |
98.50 |
95.38 |
94.61 |
93.67 |
98.55 |
91.00 |
88.64 |
86.91 |
81.12 |
85.45 |
|
Danish-DDT |
Trankit[large] |
99.81 |
95.46 |
99.81 |
98.71 |
99.81 |
98.33 |
97.87 |
97.35 |
91.75 |
90.33 |
88.81 |
85.66 |
85.42 |
Trankit[base] |
99.79 |
95.19 |
99.79 |
98.35 |
99.79 |
97.79 |
97.19 |
97.18 |
90.41 |
88.78 |
86.94 |
82.58 |
83.38 |
|
Stanza |
99.96 |
93.57 |
99.96 |
97.75 |
99.96 |
97.38 |
96.45 |
97.32 |
86.83 |
84.19 |
81.20 |
77.13 |
78.46 |
|
Dutch-Alpino |
Trankit[large] |
99.43 |
90.65 |
99.43 |
96.67 |
95.01 |
96.55 |
94.75 |
96.39 |
94.41 |
92.49 |
89.56 |
84.22 |
85.29 |
Trankit[base] |
99.22 |
89.88 |
99.22 |
96.55 |
94.92 |
96.22 |
94.56 |
96.23 |
93.28 |
91.28 |
87.88 |
82.58 |
83.86 |
|
Stanza |
99.96 |
89.98 |
99.96 |
96.33 |
94.76 |
96.28 |
94.13 |
96.97 |
89.56 |
86.44 |
81.22 |
75.76 |
77.80 |
|
Dutch-LassySmall |
Trankit[large] |
99.36 |
92.60 |
99.36 |
96.52 |
95.57 |
96.63 |
94.99 |
97.37 |
92.25 |
89.68 |
86.51 |
82.56 |
84.19 |
Trankit[base] |
99.21 |
91.09 |
99.21 |
96.20 |
95.18 |
96.25 |
94.53 |
97.19 |
91.09 |
88.18 |
84.63 |
80.47 |
82.44 |
|
Stanza |
99.90 |
77.95 |
99.90 |
95.97 |
94.87 |
96.22 |
94.05 |
97.59 |
85.34 |
81.93 |
75.54 |
71.98 |
73.49 |
|
English-EWT |
Trankit[large] |
98.67 |
90.49 |
98.67 |
96.65 |
96.47 |
96.75 |
95.49 |
97.25 |
91.29 |
89.40 |
87.41 |
83.45 |
85.73 |
Trankit[base] |
98.48 |
88.35 |
98.48 |
95.95 |
95.71 |
96.26 |
94.62 |
96.84 |
90.14 |
87.96 |
85.75 |
81.02 |
83.83 |
|
Stanza |
99.01 |
81.13 |
99.01 |
95.40 |
95.12 |
96.11 |
93.90 |
97.21 |
86.22 |
83.59 |
80.21 |
76.02 |
78.50 |
|
English-GUM |
Trankit[large] |
99.52 |
91.60 |
99.52 |
96.66 |
96.51 |
97.47 |
95.77 |
96.63 |
91.61 |
89.09 |
85.58 |
81.29 |
81.80 |
Trankit[base] |
99.45 |
91.63 |
99.45 |
96.39 |
96.24 |
97.19 |
95.46 |
96.55 |
91.04 |
88.43 |
84.80 |
80.19 |
80.81 |
|
Stanza |
99.82 |
86.35 |
99.82 |
95.89 |
95.91 |
96.87 |
94.99 |
96.80 |
87.06 |
83.57 |
78.42 |
74.68 |
74.97 |
|
English-LinES |
Trankit[large] |
99.46 |
91.87 |
99.46 |
97.31 |
95.91 |
96.96 |
93.91 |
97.98 |
89.99 |
87.00 |
85.67 |
80.34 |
83.77 |
Trankit[base] |
99.53 |
93.01 |
99.53 |
97.14 |
95.54 |
96.67 |
93.17 |
98.01 |
89.39 |
86.32 |
84.75 |
78.96 |
82.78 |
|
Stanza |
99.95 |
88.49 |
99.95 |
96.88 |
95.18 |
96.76 |
93.11 |
98.32 |
85.82 |
81.97 |
79.04 |
74.47 |
77.31 |
|
English-ParTUT |
Trankit[large] |
99.71 |
100.00 |
99.65 |
96.86 |
96.65 |
95.77 |
94.63 |
97.71 |
93.51 |
91.10 |
87.37 |
81.25 |
85.06 |
Trankit[base] |
99.66 |
100.00 |
99.60 |
96.79 |
96.55 |
95.94 |
94.67 |
97.64 |
93.15 |
90.95 |
87.21 |
81.37 |
84.96 |
|
Stanza |
99.68 |
100.00 |
99.59 |
96.15 |
95.83 |
95.21 |
93.92 |
97.45 |
90.31 |
87.35 |
82.56 |
76.19 |
80.53 |
|
Estonian-EDT |
Trankit[large] |
99.75 |
96.58 |
99.75 |
97.87 |
98.35 |
97.10 |
96.04 |
96.09 |
91.71 |
89.52 |
88.57 |
84.68 |
84.10 |
Trankit[base] |
99.72 |
96.55 |
99.72 |
97.53 |
98.13 |
96.56 |
95.37 |
95.98 |
90.65 |
88.31 |
87.15 |
82.82 |
82.81 |
|
Stanza |
99.96 |
93.32 |
99.96 |
97.19 |
98.04 |
95.77 |
94.43 |
96.05 |
86.68 |
83.82 |
82.41 |
77.63 |
78.32 |
|
Estonian-EWT |
Trankit[large] |
97.76 |
82.58 |
97.76 |
94.26 |
94.93 |
91.49 |
89.91 |
85.71 |
82.18 |
78.49 |
76.41 |
69.32 |
64.00 |
Trankit[base] |
96.96 |
83.72 |
96.96 |
92.07 |
93.16 |
89.17 |
86.89 |
84.65 |
78.21 |
73.79 |
71.59 |
62.91 |
59.90 |
|
Stanza |
99.20 |
67.14 |
99.20 |
88.86 |
91.70 |
87.16 |
83.43 |
85.62 |
67.23 |
60.07 |
56.21 |
48.32 |
47.38 |
|
Finnish-FTB |
Trankit[large] |
99.84 |
97.36 |
99.83 |
98.32 |
97.29 |
98.09 |
96.87 |
96.94 |
95.84 |
94.53 |
93.54 |
90.68 |
90.39 |
Trankit[base] |
99.75 |
95.83 |
99.74 |
97.46 |
96.23 |
97.22 |
95.61 |
96.58 |
94.17 |
92.43 |
90.84 |
87.09 |
87.79 |
|
Stanza |
100.00 |
89.59 |
99.97 |
95.50 |
95.12 |
96.51 |
93.92 |
96.16 |
89.09 |
86.39 |
83.80 |
79.90 |
81.02 |
|
Finnish-TDT |
Trankit[large] |
99.71 |
97.22 |
99.72 |
98.48 |
98.78 |
96.84 |
96.33 |
95.59 |
94.98 |
93.77 |
92.92 |
88.97 |
88.02 |
Trankit[base] |
99.62 |
95.98 |
99.62 |
97.99 |
98.44 |
96.52 |
95.76 |
95.39 |
93.47 |
91.94 |
90.78 |
86.55 |
86.00 |
|
Stanza |
99.77 |
93.05 |
99.73 |
96.97 |
97.72 |
95.36 |
94.44 |
94.98 |
88.62 |
86.18 |
84.66 |
79.73 |
80.24 |
|
French-GSD |
Trankit[large] |
99.77 |
98.80 |
99.72 |
97.89 |
99.72 |
97.34 |
96.66 |
97.90 |
94.97 |
93.37 |
90.17 |
86.35 |
87.73 |
Trankit[base] |
99.70 |
96.63 |
99.66 |
97.85 |
99.66 |
97.16 |
96.60 |
97.80 |
94.00 |
92.34 |
88.66 |
84.76 |
86.08 |
|
Stanza |
99.68 |
94.92 |
99.48 |
97.30 |
99.47 |
96.72 |
96.05 |
97.64 |
91.38 |
89.05 |
84.38 |
80.30 |
82.40 |
|
French-ParTUT |
Trankit[large] |
99.76 |
98.63 |
99.65 |
97.66 |
97.35 |
94.55 |
93.82 |
96.09 |
95.05 |
93.32 |
90.65 |
81.49 |
84.88 |
Trankit[base] |
99.74 |
98.63 |
99.69 |
97.77 |
97.54 |
94.20 |
93.66 |
96.01 |
94.20 |
92.67 |
89.26 |
78.71 |
83.56 |
|
Stanza |
99.82 |
100.00 |
99.37 |
96.60 |
96.37 |
93.98 |
93.41 |
95.48 |
90.71 |
88.37 |
83.37 |
74.41 |
77.88 |
|
French-Sequoia |
Trankit[large] |
99.81 |
94.07 |
99.78 |
99.22 |
99.78 |
98.43 |
98.13 |
98.64 |
95.70 |
94.85 |
92.95 |
90.47 |
91.15 |
Trankit[base] |
99.73 |
94.36 |
99.73 |
98.90 |
99.73 |
97.98 |
97.57 |
98.47 |
94.68 |
93.59 |
91.26 |
88.27 |
89.44 |
|
Stanza |
99.90 |
88.79 |
99.58 |
98.19 |
99.58 |
97.58 |
96.94 |
98.25 |
90.47 |
88.34 |
84.71 |
81.77 |
83.31 |
|
French-Spoken |
Trankit[large] |
99.36 |
53.06 |
99.19 |
96.80 |
96.91 |
99.19 |
94.66 |
96.34 |
85.70 |
81.84 |
76.29 |
73.65 |
73.78 |
Trankit[base] |
99.38 |
39.39 |
99.18 |
96.73 |
96.73 |
99.18 |
94.43 |
96.38 |
82.40 |
78.35 |
71.68 |
69.01 |
69.49 |
|
Stanza |
100.00 |
22.09 |
99.45 |
95.49 |
97.06 |
99.45 |
93.23 |
96.53 |
75.82 |
70.71 |
62.13 |
59.57 |
60.44 |
|
Galician-CTG |
Trankit[large] |
99.76 |
98.44 |
99.31 |
97.30 |
97.05 |
99.17 |
96.77 |
98.07 |
85.70 |
83.14 |
78.24 |
72.41 |
76.90 |
Trankit[base] |
99.76 |
98.09 |
99.38 |
97.17 |
96.83 |
99.23 |
96.54 |
98.06 |
85.51 |
82.81 |
77.50 |
71.49 |
76.20 |
|
Stanza |
99.89 |
99.13 |
99.32 |
97.21 |
96.99 |
99.14 |
96.71 |
97.94 |
85.22 |
82.66 |
77.24 |
71.13 |
75.96 |
|
Galician-TreeGal |
Trankit[large] |
99.47 |
95.52 |
99.22 |
97.62 |
95.68 |
96.50 |
94.97 |
91.08 |
87.17 |
83.90 |
80.31 |
74.67 |
68.39 |
Trankit[base] |
99.47 |
94.60 |
99.06 |
97.06 |
94.90 |
95.89 |
94.08 |
90.91 |
85.38 |
81.96 |
77.96 |
71.57 |
66.32 |
|
Stanza |
99.59 |
89.17 |
98.41 |
94.29 |
91.81 |
93.36 |
90.88 |
94.39 |
78.04 |
72.94 |
65.61 |
59.06 |
61.49 |
|
German-GSD |
Trankit[large] |
99.71 |
89.72 |
99.72 |
95.23 |
97.68 |
91.68 |
87.21 |
96.58 |
89.01 |
85.20 |
81.49 |
65.82 |
77.20 |
Trankit[base] |
99.75 |
92.72 |
99.75 |
95.04 |
97.57 |
91.51 |
86.86 |
96.60 |
88.73 |
84.77 |
80.78 |
64.76 |
76.58 |
|
Stanza |
99.53 |
85.79 |
99.53 |
94.07 |
96.98 |
89.52 |
84.51 |
96.37 |
85.39 |
80.61 |
75.38 |
58.57 |
71.24 |
|
German-HDT |
Trankit[large] |
99.92 |
99.67 |
99.92 |
98.44 |
98.41 |
94.05 |
93.70 |
97.36 |
97.63 |
96.86 |
95.14 |
85.73 |
91.67 |
Trankit[base] |
99.90 |
99.50 |
99.90 |
98.42 |
98.37 |
93.95 |
93.52 |
97.35 |
97.38 |
96.51 |
94.63 |
85.02 |
91.18 |
|
Stanza |
100.00 |
97.41 |
100.00 |
98.04 |
97.94 |
91.77 |
91.34 |
97.48 |
94.91 |
92.59 |
88.73 |
77.26 |
85.63 |
|
Greek-GDT |
Trankit[large] |
99.85 |
93.50 |
99.85 |
98.41 |
98.41 |
96.34 |
95.84 |
96.73 |
95.25 |
93.87 |
91.74 |
85.40 |
86.54 |
Trankit[base] |
99.75 |
93.57 |
99.75 |
98.04 |
98.04 |
95.41 |
94.73 |
96.55 |
94.16 |
92.80 |
89.84 |
82.39 |
84.83 |
|
Stanza |
99.88 |
93.18 |
99.89 |
97.84 |
97.84 |
94.94 |
94.33 |
96.49 |
91.12 |
88.78 |
84.12 |
78.00 |
79.48 |
|
Hebrew-HTB |
Trankit[large] |
99.81 |
99.69 |
96.31 |
94.32 |
94.32 |
93.03 |
92.39 |
93.48 |
88.41 |
86.04 |
82.23 |
74.92 |
78.35 |
Trankit[base] |
99.79 |
100.00 |
96.03 |
93.75 |
93.75 |
91.96 |
91.24 |
93.21 |
87.02 |
84.55 |
80.34 |
72.38 |
76.52 |
|
Stanza |
99.98 |
99.69 |
93.19 |
90.46 |
90.46 |
89.24 |
88.45 |
90.27 |
79.18 |
76.60 |
71.05 |
64.51 |
67.79 |
|
Hindi-HDTB |
Trankit[large] |
99.88 |
99.91 |
99.88 |
98.01 |
97.70 |
93.91 |
92.38 |
96.54 |
95.95 |
92.96 |
89.79 |
79.69 |
88.47 |
Trankit[base] |
99.89 |
99.64 |
99.89 |
97.77 |
97.38 |
94.03 |
92.33 |
96.54 |
95.68 |
92.70 |
89.59 |
79.60 |
88.28 |
|
Stanza |
100.00 |
99.44 |
100.00 |
97.59 |
97.08 |
94.03 |
92.11 |
96.66 |
94.80 |
91.74 |
88.20 |
78.73 |
87.01 |
|
Hungarian-Szeged |
Trankit[large] |
99.59 |
99.33 |
99.59 |
97.49 |
99.59 |
95.23 |
94.40 |
94.45 |
91.31 |
87.78 |
86.83 |
78.95 |
80.31 |
Trankit[base] |
99.41 |
98.00 |
99.41 |
96.97 |
99.41 |
94.47 |
93.47 |
94.28 |
89.43 |
85.70 |
85.08 |
76.13 |
78.73 |
|
Stanza |
99.87 |
97.00 |
99.87 |
96.03 |
99.87 |
93.76 |
92.94 |
94.25 |
83.62 |
78.86 |
77.14 |
69.46 |
71.87 |
|
Indonesian-GSD |
Trankit[large] |
99.89 |
95.54 |
99.89 |
93.39 |
95.06 |
96.11 |
89.22 |
99.53 |
86.33 |
79.81 |
77.99 |
69.08 |
77.60 |
Trankit[base] |
99.86 |
95.37 |
99.86 |
93.57 |
94.18 |
95.67 |
88.65 |
99.49 |
86.55 |
80.28 |
78.64 |
69.42 |
78.26 |
|
Stanza |
99.99 |
93.78 |
99.99 |
93.68 |
94.79 |
96.00 |
89.17 |
99.61 |
85.17 |
79.19 |
77.04 |
68.86 |
76.68 |
|
Irish-IDT |
Trankit[large] |
99.47 |
98.24 |
99.47 |
94.72 |
93.74 |
80.90 |
77.94 |
92.64 |
83.47 |
76.86 |
70.64 |
48.55 |
64.06 |
Trankit[base] |
99.32 |
97.25 |
99.32 |
93.88 |
92.46 |
80.36 |
76.72 |
92.34 |
82.52 |
74.91 |
67.96 |
46.29 |
61.34 |
|
Stanza |
99.76 |
95.93 |
99.76 |
93.90 |
92.43 |
78.19 |
75.00 |
91.79 |
82.65 |
74.03 |
66.11 |
42.98 |
59.09 |
|
Italian-ISDT |
Trankit[large] |
99.88 |
99.07 |
99.86 |
98.72 |
98.63 |
98.32 |
97.79 |
98.33 |
95.73 |
94.45 |
91.97 |
89.08 |
89.45 |
Trankit[base] |
99.88 |
98.76 |
99.87 |
98.58 |
98.46 |
98.20 |
97.60 |
98.23 |
95.31 |
93.87 |
90.93 |
87.81 |
88.45 |
|
Stanza |
99.91 |
98.76 |
99.76 |
98.01 |
97.91 |
97.72 |
97.11 |
98.10 |
92.79 |
90.84 |
86.43 |
83.60 |
84.23 |
|
Italian-ParTUT |
Trankit[large] |
99.81 |
100.00 |
99.79 |
98.58 |
98.42 |
98.15 |
97.54 |
97.84 |
96.19 |
94.11 |
90.65 |
87.75 |
87.86 |
Trankit[base] |
99.82 |
100.00 |
99.81 |
98.63 |
98.41 |
98.16 |
97.47 |
97.94 |
95.38 |
93.32 |
89.28 |
86.22 |
86.55 |
|
Stanza |
99.81 |
100.00 |
99.77 |
97.82 |
97.76 |
97.79 |
96.94 |
97.57 |
92.24 |
90.01 |
84.39 |
81.77 |
82.05 |
|
Italian-PoSTWITA |
Trankit[large] |
99.34 |
73.95 |
99.18 |
96.60 |
96.43 |
96.52 |
95.31 |
96.41 |
86.33 |
82.54 |
78.49 |
74.27 |
75.83 |
Trankit[base] |
99.29 |
69.95 |
99.07 |
96.10 |
95.91 |
95.87 |
94.53 |
96.30 |
84.19 |
80.32 |
75.33 |
71.09 |
72.98 |
|
Stanza |
99.71 |
63.70 |
99.46 |
96.19 |
96.04 |
96.28 |
95.01 |
96.70 |
82.67 |
78.27 |
72.20 |
68.55 |
70.35 |
|
Italian-TWITTIRO |
Trankit[large] |
99.15 |
65.72 |
98.89 |
95.47 |
94.90 |
94.09 |
91.98 |
93.12 |
84.73 |
79.86 |
73.94 |
66.46 |
66.58 |
Trankit[base] |
99.22 |
56.00 |
99.01 |
95.31 |
94.74 |
93.83 |
91.68 |
92.96 |
83.44 |
78.30 |
70.79 |
63.25 |
63.81 |
|
Stanza |
99.34 |
52.40 |
98.76 |
94.41 |
94.01 |
93.34 |
91.45 |
93.17 |
78.87 |
72.85 |
64.64 |
58.67 |
59.35 |
|
Italian-VIT |
Trankit[large] |
99.97 |
98.18 |
99.84 |
98.07 |
97.29 |
97.76 |
96.16 |
98.42 |
93.02 |
90.44 |
86.85 |
82.53 |
84.91 |
Trankit[base] |
99.99 |
96.52 |
99.81 |
97.82 |
97.02 |
97.39 |
95.74 |
98.31 |
92.39 |
89.60 |
85.59 |
80.70 |
83.64 |
|
Stanza |
99.98 |
94.92 |
99.49 |
97.21 |
96.23 |
96.79 |
94.99 |
98.01 |
89.32 |
85.87 |
80.26 |
76.16 |
78.61 |
|
Japanese-GSD |
Trankit[large] |
95.25 |
95.88 |
95.25 |
93.66 |
93.47 |
95.23 |
93.44 |
94.68 |
86.67 |
85.56 |
78.00 |
76.02 |
77.62 |
Trankit[base] |
94.57 |
95.49 |
94.57 |
92.86 |
92.44 |
94.56 |
92.42 |
93.99 |
84.58 |
83.38 |
75.60 |
73.67 |
75.14 |
|
Stanza |
92.67 |
94.57 |
92.67 |
91.16 |
90.84 |
92.66 |
90.84 |
92.02 |
81.20 |
80.16 |
71.39 |
69.85 |
71.01 |
|
Kazakh-KTB |
Trankit[large] |
95.98 |
81.71 |
95.37 |
77.94 |
77.47 |
63.01 |
55.46 |
50.60 |
47.46 |
37.98 |
36.01 |
19.00 |
12.05 |
Trankit[base] |
94.48 |
90.00 |
93.62 |
75.94 |
75.67 |
62.28 |
54.51 |
49.76 |
46.42 |
36.84 |
34.72 |
18.65 |
11.81 |
|
Stanza |
93.46 |
88.56 |
94.16 |
56.23 |
56.10 |
42.73 |
36.96 |
52.12 |
44.33 |
25.21 |
20.28 |
7.63 |
10.01 |
|
Korean-GSD |
Trankit[large] |
98.57 |
98.08 |
98.57 |
95.71 |
90.88 |
98.35 |
88.90 |
91.93 |
89.87 |
87.22 |
85.97 |
83.63 |
79.65 |
Trankit[base] |
98.63 |
97.67 |
98.63 |
95.63 |
90.32 |
98.43 |
88.26 |
91.96 |
88.48 |
85.77 |
84.26 |
81.98 |
78.08 |
|
Stanza |
99.88 |
96.65 |
99.88 |
96.18 |
90.14 |
99.66 |
88.00 |
92.69 |
87.29 |
83.53 |
81.34 |
79.29 |
75.31 |
|
Korean-Kaist |
Trankit[large] |
98.70 |
99.87 |
98.70 |
95.13 |
88.07 |
98.70 |
88.07 |
92.36 |
90.00 |
88.22 |
86.37 |
83.56 |
80.16 |
Trankit[base] |
98.79 |
99.14 |
98.79 |
94.99 |
87.62 |
98.79 |
87.62 |
92.44 |
88.72 |
86.96 |
84.99 |
81.84 |
78.90 |
|
Stanza |
100.00 |
99.93 |
100.00 |
95.45 |
86.31 |
100.00 |
86.31 |
93.02 |
88.41 |
86.38 |
83.95 |
80.63 |
77.57 |
|
Kurmanji-MG |
Trankit[large] |
94.95 |
91.50 |
94.63 |
75.07 |
74.16 |
57.15 |
52.27 |
57.63 |
37.12 |
29.89 |
25.57 |
9.04 |
10.16 |
Trankit[base] |
94.52 |
80.56 |
94.20 |
74.33 |
73.44 |
56.54 |
51.38 |
57.61 |
35.65 |
28.58 |
25.35 |
8.88 |
10.76 |
|
Stanza |
94.81 |
87.43 |
94.49 |
57.17 |
55.91 |
43.02 |
38.41 |
56.13 |
32.01 |
21.91 |
16.35 |
3.84 |
5.84 |
|
Latin-ITTB |
Trankit[large] |
100.00 |
94.54 |
100.00 |
98.97 |
97.29 |
97.98 |
96.41 |
99.13 |
93.25 |
91.87 |
90.75 |
87.86 |
90.00 |
Trankit[base] |
100.00 |
94.57 |
100.00 |
98.76 |
96.74 |
97.54 |
95.68 |
99.07 |
92.42 |
90.91 |
89.45 |
86.12 |
88.71 |
|
Stanza |
99.99 |
80.66 |
99.99 |
98.09 |
95.38 |
96.43 |
93.80 |
98.90 |
87.61 |
85.36 |
84.23 |
80.28 |
83.60 |
|
Latin-Perseus |
Trankit[large] |
99.60 |
97.93 |
99.60 |
92.84 |
83.33 |
86.79 |
82.33 |
70.34 |
83.50 |
76.76 |
73.58 |
60.70 |
44.41 |
Trankit[base] |
99.45 |
97.87 |
99.45 |
90.15 |
77.12 |
81.12 |
75.64 |
69.95 |
78.01 |
69.58 |
65.24 |
49.58 |
40.23 |
|
Stanza |
100.00 |
98.24 |
100.00 |
90.63 |
78.42 |
82.42 |
77.74 |
83.08 |
71.94 |
61.99 |
57.89 |
45.28 |
47.28 |
|
Latin-PROIEL |
Trankit[large] |
99.85 |
66.10 |
99.85 |
97.79 |
97.75 |
93.22 |
92.53 |
97.21 |
86.43 |
83.33 |
81.62 |
73.62 |
79.55 |
Trankit[base] |
99.82 |
58.16 |
99.82 |
96.80 |
96.83 |
91.28 |
90.27 |
96.88 |
82.23 |
78.58 |
76.36 |
67.10 |
74.43 |
|
Stanza |
100.00 |
43.04 |
100.00 |
96.92 |
97.10 |
91.24 |
90.32 |
96.78 |
76.55 |
72.37 |
70.06 |
61.28 |
68.19 |
|
Latvian-LVTB |
Trankit[large] |
99.73 |
98.69 |
99.73 |
97.61 |
91.22 |
95.18 |
90.72 |
95.83 |
93.63 |
91.25 |
89.78 |
82.69 |
85.58 |
Trankit[base] |
99.71 |
99.10 |
99.71 |
97.16 |
90.24 |
94.47 |
89.62 |
95.61 |
92.05 |
89.44 |
87.73 |
79.78 |
83.52 |
|
Stanza |
99.82 |
99.01 |
99.82 |
96.03 |
88.25 |
93.46 |
87.73 |
95.55 |
87.84 |
84.44 |
82.16 |
73.91 |
78.25 |
|
Lithuanian-ALKSNIS |
Trankit[large] |
99.84 |
95.72 |
99.84 |
97.45 |
93.98 |
94.46 |
93.30 |
94.30 |
90.48 |
87.67 |
86.66 |
79.86 |
80.20 |
Trankit[base] |
99.82 |
95.10 |
99.82 |
97.03 |
92.35 |
93.00 |
91.54 |
94.05 |
88.30 |
84.96 |
83.59 |
75.11 |
77.35 |
|
Stanza |
99.87 |
88.79 |
99.87 |
93.37 |
85.67 |
87.84 |
84.84 |
92.51 |
78.54 |
73.11 |
70.66 |
60.81 |
65.53 |
|
Lithuanian-HSE |
Trankit[large] |
97.71 |
100.00 |
97.71 |
90.59 |
89.85 |
79.64 |
75.90 |
80.02 |
71.41 |
62.05 |
59.15 |
41.13 |
44.82 |
Trankit[base] |
98.22 |
94.55 |
98.22 |
90.46 |
89.71 |
77.92 |
74.18 |
80.07 |
66.70 |
58.47 |
55.18 |
36.60 |
40.03 |
|
Stanza |
97.53 |
51.11 |
97.53 |
81.08 |
80.04 |
70.72 |
66.44 |
76.90 |
48.10 |
37.45 |
32.37 |
21.10 |
24.86 |
|
Marathi-UFAL |
Trankit[large] |
99.20 |
69.31 |
97.22 |
87.79 |
97.22 |
70.62 |
67.47 |
81.50 |
72.79 |
63.36 |
59.67 |
36.63 |
46.50 |
Trankit[base] |
99.20 |
60.87 |
95.25 |
82.83 |
95.25 |
69.43 |
66.02 |
79.17 |
60.90 |
54.08 |
52.19 |
28.81 |
40.50 |
|
Stanza |
98.00 |
76.40 |
92.25 |
77.24 |
92.25 |
60.27 |
58.55 |
75.77 |
66.42 |
52.64 |
42.80 |
24.15 |
33.90 |
|
Norwegian_Nynorsk-Nynorsk |
Trankit[large] |
99.84 |
98.97 |
99.84 |
98.52 |
99.84 |
97.79 |
97.13 |
98.01 |
95.23 |
93.82 |
92.35 |
89.01 |
89.79 |
Trankit[base] |
99.81 |
98.71 |
99.81 |
98.20 |
99.81 |
97.20 |
96.48 |
97.89 |
94.15 |
92.58 |
90.70 |
86.61 |
88.15 |
|
Stanza |
99.97 |
94.85 |
99.97 |
97.92 |
99.97 |
96.88 |
96.03 |
97.90 |
91.87 |
89.73 |
87.28 |
82.86 |
84.78 |
|
Norwegian_Nynorsk-NynorskLIA |
Trankit[large] |
99.76 |
99.53 |
99.76 |
96.48 |
99.76 |
95.59 |
93.57 |
97.49 |
81.96 |
77.85 |
73.57 |
67.19 |
71.00 |
Trankit[base] |
99.74 |
99.53 |
99.74 |
96.31 |
99.74 |
95.41 |
93.29 |
97.50 |
80.86 |
76.44 |
71.96 |
65.82 |
69.71 |
|
Stanza |
100.00 |
99.69 |
100.00 |
95.92 |
100.00 |
94.82 |
92.70 |
97.72 |
77.82 |
72.94 |
67.56 |
61.32 |
65.54 |
|
Norwegian-Bokmaal |
Trankit[large] |
99.88 |
98.89 |
99.88 |
98.85 |
99.88 |
98.07 |
97.61 |
98.40 |
95.54 |
94.33 |
92.82 |
90.15 |
90.84 |
Trankit[base] |
99.88 |
99.20 |
99.88 |
98.66 |
99.88 |
97.60 |
97.02 |
98.34 |
94.78 |
93.47 |
91.77 |
88.29 |
89.72 |
|
Stanza |
99.99 |
97.17 |
99.99 |
98.29 |
99.99 |
97.17 |
96.41 |
98.36 |
92.57 |
90.69 |
88.32 |
84.41 |
86.33 |
|
Old_French-SRCMF |
Trankit[large] |
99.91 |
100.00 |
99.91 |
96.96 |
96.83 |
98.32 |
96.45 |
99.91 |
94.30 |
90.75 |
88.69 |
85.43 |
88.69 |
Trankit[base] |
99.84 |
100.00 |
99.84 |
96.36 |
96.21 |
97.75 |
95.72 |
99.84 |
92.82 |
88.76 |
86.12 |
82.63 |
86.12 |
|
Stanza |
100.00 |
100.00 |
100.00 |
96.05 |
96.09 |
97.74 |
95.56 |
100.00 |
91.38 |
86.35 |
83.39 |
80.05 |
83.39 |
|
Old_Russian-TOROT |
Trankit[large] |
98.87 |
51.91 |
98.87 |
94.70 |
94.63 |
89.61 |
88.02 |
90.87 |
78.64 |
74.60 |
71.82 |
63.30 |
66.49 |
Trankit[base] |
98.44 |
42.22 |
98.44 |
92.63 |
92.66 |
86.75 |
84.52 |
90.00 |
74.14 |
68.92 |
65.57 |
55.81 |
60.56 |
|
Stanza |
100.00 |
35.69 |
100.00 |
93.63 |
93.83 |
86.76 |
84.80 |
91.35 |
72.94 |
67.00 |
63.60 |
54.13 |
59.18 |
|
Persian-Seraji |
Trankit[large] |
99.26 |
99.25 |
99.20 |
97.78 |
97.67 |
97.70 |
97.35 |
97.35 |
92.24 |
89.58 |
86.86 |
84.97 |
84.90 |
Trankit[base] |
99.22 |
99.25 |
99.11 |
97.35 |
97.24 |
97.36 |
96.90 |
97.29 |
91.38 |
88.68 |
85.92 |
83.86 |
84.08 |
|
Stanza |
100.00 |
99.25 |
99.65 |
97.29 |
97.30 |
97.37 |
96.86 |
97.73 |
89.45 |
86.06 |
82.78 |
81.00 |
81.08 |
|
Polish-LFG |
Trankit[large] |
98.34 |
99.57 |
98.34 |
97.84 |
95.52 |
96.00 |
95.05 |
95.48 |
93.90 |
93.04 |
92.58 |
89.18 |
88.61 |
Trankit[base] |
98.32 |
99.91 |
98.32 |
97.66 |
94.59 |
95.05 |
94.00 |
95.37 |
93.31 |
92.17 |
91.43 |
86.88 |
87.55 |
|
Stanza |
99.95 |
99.83 |
99.95 |
98.55 |
94.66 |
95.84 |
94.07 |
96.86 |
95.80 |
93.94 |
92.35 |
87.62 |
88.64 |
|
Polish-PDB |
Trankit[large] |
99.93 |
98.71 |
99.92 |
99.16 |
96.92 |
97.11 |
96.54 |
97.56 |
96.43 |
94.88 |
93.77 |
89.78 |
90.53 |
Trankit[base] |
99.91 |
98.53 |
99.89 |
99.06 |
96.29 |
96.44 |
95.77 |
97.52 |
95.52 |
93.86 |
92.50 |
87.67 |
89.34 |
|
Stanza |
99.87 |
98.39 |
99.83 |
98.31 |
94.04 |
94.27 |
93.13 |
97.29 |
92.68 |
90.40 |
88.35 |
81.69 |
85.42 |
|
Portuguese-Bosque |
Trankit[large] |
99.75 |
97.18 |
99.67 |
97.52 |
99.67 |
96.50 |
95.17 |
97.97 |
93.31 |
90.91 |
87.75 |
81.09 |
85.36 |
Trankit[base] |
99.70 |
97.48 |
99.65 |
97.27 |
99.65 |
96.50 |
94.95 |
97.89 |
92.76 |
90.25 |
86.96 |
80.03 |
84.52 |
|
Stanza |
99.77 |
94.30 |
99.67 |
97.04 |
99.67 |
96.36 |
94.91 |
97.80 |
90.67 |
87.57 |
82.59 |
76.78 |
80.30 |
|
Portuguese-GSD |
Trankit[large] |
99.81 |
97.10 |
99.72 |
98.43 |
98.43 |
99.61 |
98.41 |
99.20 |
95.35 |
94.37 |
92.23 |
90.46 |
91.47 |
Trankit[base] |
99.82 |
96.76 |
99.71 |
98.30 |
98.30 |
99.61 |
98.28 |
99.19 |
94.92 |
93.95 |
91.65 |
89.58 |
90.89 |
|
Stanza |
99.96 |
98.00 |
99.87 |
98.18 |
98.18 |
99.79 |
98.17 |
95.83 |
92.83 |
91.36 |
87.44 |
85.87 |
86.75 |
|
Romanian-Nonstandard |
Trankit[large] |
98.74 |
98.00 |
98.74 |
96.23 |
91.64 |
90.51 |
89.07 |
94.66 |
90.68 |
86.89 |
83.13 |
70.10 |
78.50 |
Trankit[base] |
98.68 |
98.57 |
98.68 |
96.04 |
91.48 |
90.33 |
88.89 |
94.57 |
90.14 |
86.40 |
82.40 |
69.46 |
77.93 |
|
Stanza |
98.96 |
97.53 |
98.96 |
95.40 |
90.73 |
89.79 |
88.19 |
94.63 |
87.24 |
82.71 |
77.60 |
65.24 |
73.52 |
|
Romanian-RRT |
Trankit[large] |
99.60 |
98.49 |
99.60 |
97.90 |
97.35 |
97.43 |
97.11 |
97.98 |
93.60 |
89.50 |
86.61 |
82.78 |
84.60 |
Trankit[base] |
99.72 |
97.67 |
99.72 |
97.87 |
97.25 |
97.44 |
97.01 |
98.05 |
93.14 |
89.04 |
85.93 |
82.02 |
84.01 |
|
Stanza |
99.77 |
96.64 |
99.77 |
97.54 |
96.97 |
97.13 |
96.75 |
97.95 |
90.66 |
85.85 |
81.49 |
77.94 |
79.84 |
|
Russian-GSD |
Trankit[large] |
99.79 |
99.25 |
99.79 |
98.25 |
97.89 |
95.78 |
94.73 |
95.75 |
92.70 |
89.75 |
88.71 |
82.92 |
83.66 |
Trankit[base] |
99.63 |
98.25 |
99.63 |
97.96 |
97.65 |
94.86 |
93.83 |
95.50 |
91.86 |
88.62 |
87.41 |
80.83 |
82.36 |
|
Stanza |
99.65 |
97.16 |
99.65 |
97.38 |
97.18 |
93.11 |
92.22 |
95.34 |
88.97 |
84.83 |
82.37 |
75.16 |
77.75 |
|
Russian-SynTagRus |
Trankit[large] |
99.71 |
99.45 |
99.71 |
99.06 |
99.71 |
98.20 |
97.99 |
97.99 |
95.66 |
94.65 |
93.78 |
91.68 |
91.51 |
Trankit[base] |
99.71 |
99.14 |
99.71 |
98.94 |
99.71 |
97.85 |
97.59 |
97.89 |
95.19 |
94.08 |
93.13 |
90.59 |
90.77 |
|
Stanza |
99.57 |
98.86 |
99.57 |
98.20 |
99.57 |
95.91 |
95.59 |
97.51 |
92.38 |
90.60 |
89.01 |
85.04 |
86.78 |
|
Russian-Taiga |
Trankit[large] |
98.90 |
92.32 |
98.90 |
96.16 |
96.62 |
91.02 |
87.69 |
91.90 |
85.25 |
81.73 |
80.06 |
68.48 |
71.59 |
Trankit[base] |
98.77 |
92.60 |
98.77 |
95.50 |
97.27 |
89.42 |
86.58 |
91.46 |
83.08 |
79.15 |
76.91 |
64.25 |
68.53 |
|
Stanza |
97.11 |
85.79 |
97.11 |
92.25 |
94.70 |
85.76 |
82.61 |
89.28 |
72.09 |
66.00 |
61.80 |
51.94 |
55.64 |
|
Scottish_Gaelic-ARCOSG |
Trankit[large] |
99.43 |
57.46 |
99.42 |
94.70 |
88.65 |
90.71 |
87.13 |
95.48 |
81.60 |
76.46 |
70.62 |
61.71 |
66.95 |
Trankit[base] |
99.26 |
54.10 |
99.25 |
92.98 |
85.47 |
88.25 |
83.78 |
95.06 |
79.48 |
73.09 |
66.41 |
56.27 |
62.83 |
|
Stanza |
99.48 |
55.35 |
99.47 |
92.50 |
84.89 |
87.99 |
83.93 |
95.51 |
77.90 |
70.81 |
62.63 |
54.00 |
59.74 |
|
Serbian-SET |
Trankit[large] |
99.91 |
100.00 |
99.91 |
99.06 |
96.22 |
96.40 |
95.84 |
96.90 |
95.57 |
93.13 |
91.92 |
85.76 |
87.71 |
Trankit[base] |
99.91 |
99.71 |
99.91 |
98.97 |
95.82 |
95.96 |
95.32 |
96.90 |
95.24 |
92.94 |
91.53 |
84.84 |
87.46 |
|
Stanza |
100.00 |
99.33 |
100.00 |
98.44 |
94.26 |
94.55 |
93.86 |
96.34 |
91.79 |
88.78 |
86.50 |
79.48 |
82.38 |
|
Simplified_Chinese-GSDSimp |
Trankit[large] |
97.66 |
98.20 |
97.66 |
94.99 |
94.78 |
97.24 |
94.32 |
97.66 |
87.01 |
84.63 |
83.23 |
79.84 |
83.23 |
Trankit[base] |
96.94 |
99.70 |
96.94 |
94.17 |
93.98 |
96.51 |
93.52 |
96.94 |
84.64 |
81.96 |
80.14 |
76.30 |
80.14 |
|
Stanza |
92.92 |
99.10 |
92.92 |
89.05 |
88.84 |
92.12 |
88.03 |
92.92 |
73.44 |
70.44 |
67.69 |
64.07 |
67.69 |
|
Slovak-SNK |
Trankit[large] |
99.94 |
98.49 |
99.94 |
97.90 |
90.82 |
95.22 |
90.20 |
94.79 |
96.97 |
95.60 |
95.04 |
87.64 |
88.39 |
Trankit[base] |
99.93 |
98.07 |
99.93 |
97.80 |
89.02 |
94.00 |
88.38 |
94.66 |
95.72 |
93.97 |
93.19 |
84.33 |
86.63 |
|
Stanza |
99.97 |
90.93 |
99.97 |
96.34 |
87.15 |
91.59 |
86.34 |
94.73 |
89.96 |
86.82 |
84.74 |
75.39 |
79.35 |
|
Slovenian-SSJ |
Trankit[large] |
99.97 |
100.00 |
99.97 |
99.24 |
97.83 |
97.99 |
97.60 |
97.55 |
96.91 |
96.06 |
94.88 |
91.78 |
91.37 |
Trankit[base] |
99.93 |
99.81 |
99.93 |
99.03 |
96.70 |
96.97 |
96.38 |
97.49 |
95.94 |
94.99 |
93.53 |
89.09 |
90.12 |
|
Stanza |
99.91 |
91.60 |
99.91 |
98.29 |
95.08 |
95.37 |
94.56 |
97.34 |
91.63 |
89.60 |
87.18 |
82.35 |
84.37 |
|
Slovenian-SST |
Trankit[large] |
99.84 |
33.03 |
99.84 |
95.81 |
91.74 |
91.86 |
89.70 |
88.72 |
74.86 |
71.02 |
66.85 |
59.58 |
56.40 |
Trankit[base] |
99.79 |
31.96 |
99.79 |
94.90 |
90.27 |
90.37 |
87.92 |
88.66 |
71.15 |
66.65 |
61.94 |
54.19 |
52.26 |
|
Stanza |
100.00 |
26.59 |
100.00 |
93.66 |
88.09 |
88.06 |
85.27 |
94.78 |
63.13 |
56.50 |
51.34 |
44.81 |
48.96 |
|
Spanish-AnCora |
Trankit[large] |
99.91 |
99.39 |
99.91 |
99.06 |
99.00 |
98.85 |
98.36 |
99.15 |
94.77 |
93.29 |
91.10 |
89.23 |
90.14 |
Trankit[base] |
99.94 |
99.13 |
99.93 |
99.02 |
98.94 |
98.80 |
98.27 |
99.17 |
94.11 |
92.41 |
89.66 |
87.60 |
88.71 |
|
Stanza |
99.98 |
99.07 |
99.98 |
98.78 |
98.67 |
98.59 |
97.97 |
99.19 |
92.21 |
90.01 |
86.05 |
84.22 |
85.20 |
|
Spanish-GSD |
Trankit[large] |
99.93 |
99.30 |
99.88 |
97.38 |
99.88 |
96.96 |
95.23 |
98.59 |
93.47 |
91.56 |
88.58 |
80.91 |
86.50 |
Trankit[base] |
99.91 |
98.94 |
99.88 |
97.41 |
99.88 |
96.88 |
95.23 |
98.62 |
92.66 |
90.50 |
87.01 |
79.83 |
85.04 |
|
Stanza |
99.96 |
95.97 |
99.87 |
96.69 |
99.87 |
96.40 |
94.44 |
98.44 |
89.61 |
86.73 |
81.22 |
73.96 |
79.19 |
|
Swedish-LinES |
Trankit[large] |
99.89 |
90.64 |
99.89 |
98.05 |
95.77 |
90.98 |
88.52 |
96.94 |
91.74 |
88.73 |
87.89 |
74.52 |
83.97 |
Trankit[base] |
99.73 |
90.57 |
99.73 |
97.60 |
95.23 |
90.50 |
87.93 |
96.72 |
90.45 |
87.36 |
86.11 |
72.78 |
82.24 |
|
Stanza |
99.94 |
86.99 |
99.94 |
96.97 |
94.58 |
90.11 |
87.33 |
96.79 |
87.10 |
83.06 |
80.76 |
67.97 |
77.44 |
|
Swedish-Talbanken |
Trankit[large] |
99.91 |
99.26 |
99.91 |
99.06 |
98.09 |
98.04 |
97.43 |
97.87 |
94.41 |
92.97 |
92.08 |
88.67 |
89.28 |
Trankit[base] |
99.87 |
99.38 |
99.87 |
98.76 |
97.77 |
97.73 |
97.03 |
97.82 |
93.61 |
91.87 |
90.72 |
86.97 |
88.03 |
|
Stanza |
99.97 |
98.85 |
99.97 |
97.65 |
96.57 |
96.70 |
95.63 |
97.51 |
88.96 |
85.91 |
83.59 |
79.17 |
80.78 |
|
Tamil-TTB |
Trankit[large] |
98.33 |
100.00 |
94.44 |
87.64 |
83.92 |
87.19 |
82.56 |
88.85 |
72.34 |
67.66 |
66.19 |
58.16 |
61.43 |
Trankit[base] |
98.02 |
100.00 |
93.64 |
86.18 |
82.09 |
86.43 |
80.27 |
88.09 |
68.37 |
63.67 |
61.78 |
52.73 |
57.32 |
|
Stanza |
99.58 |
95.08 |
91.42 |
82.60 |
78.80 |
81.89 |
78.10 |
85.14 |
61.23 |
55.76 |
53.43 |
46.40 |
49.61 |
|
Telugu-MTG |
Trankit[large] |
98.89 |
98.62 |
98.89 |
94.31 |
94.31 |
98.33 |
94.31 |
98.89 |
93.19 |
87.08 |
84.46 |
80.65 |
84.46 |
Trankit[base] |
98.89 |
98.62 |
98.89 |
94.32 |
94.32 |
97.92 |
94.32 |
98.89 |
91.97 |
84.35 |
81.10 |
78.44 |
81.10 |
|
Stanza |
100.00 |
97.95 |
100.00 |
92.93 |
92.93 |
99.17 |
92.93 |
100.00 |
89.32 |
79.89 |
74.88 |
71.25 |
74.88 |
|
Turkish-IMST |
Trankit[large] |
99.84 |
98.32 |
98.91 |
96.20 |
95.33 |
92.78 |
91.19 |
96.21 |
78.94 |
73.14 |
71.15 |
63.81 |
69.11 |
Trankit[base] |
99.86 |
98.18 |
98.68 |
95.15 |
94.35 |
92.02 |
89.94 |
95.80 |
76.59 |
70.75 |
68.28 |
60.61 |
66.24 |
|
Stanza |
99.89 |
97.62 |
98.07 |
94.21 |
93.43 |
92.08 |
90.27 |
94.92 |
70.78 |
64.50 |
61.62 |
56.04 |
59.60 |
|
Ukrainian-IU |
Trankit[large] |
99.77 |
97.55 |
99.76 |
98.50 |
96.32 |
96.09 |
95.35 |
96.99 |
94.63 |
93.16 |
91.78 |
86.22 |
88.03 |
Trankit[base] |
99.78 |
97.72 |
99.76 |
98.33 |
94.96 |
94.94 |
93.86 |
96.98 |
93.44 |
91.69 |
89.89 |
83.20 |
86.33 |
|
Stanza |
99.81 |
96.65 |
99.79 |
96.77 |
92.49 |
92.53 |
91.31 |
96.49 |
87.11 |
83.86 |
80.51 |
73.38 |
77.28 |
|
Urdu-UDTB |
Trankit[large] |
99.75 |
97.67 |
99.75 |
94.43 |
92.77 |
81.91 |
78.35 |
95.44 |
88.52 |
83.26 |
78.30 |
57.41 |
75.87 |
Trankit[base] |
99.66 |
98.32 |
99.66 |
94.15 |
92.66 |
83.04 |
79.29 |
95.33 |
87.81 |
82.51 |
77.31 |
57.57 |
74.83 |
|
Stanza |
100.00 |
98.88 |
100.00 |
94.42 |
92.62 |
84.21 |
80.36 |
95.62 |
88.30 |
82.78 |
77.06 |
59.48 |
74.75 |
|
Uyghur-UDT |
Trankit[large] |
97.95 |
88.95 |
97.95 |
88.50 |
91.34 |
86.56 |
79.14 |
94.45 |
80.19 |
70.14 |
66.06 |
51.63 |
62.99 |
Trankit[base] |
97.63 |
88.51 |
97.63 |
87.47 |
90.37 |
85.31 |
77.28 |
94.26 |
78.36 |
68.24 |
63.89 |
48.42 |
60.70 |
|
Stanza |
99.79 |
86.90 |
99.79 |
89.45 |
91.92 |
87.92 |
80.54 |
96.16 |
75.55 |
63.61 |
57.00 |
46.06 |
54.39 |
|
Vietnamese-VTB |
Trankit[large] |
94.88 |
96.63 |
94.88 |
89.70 |
88.14 |
94.64 |
88.07 |
94.88 |
71.07 |
65.37 |
63.67 |
60.22 |
63.67 |
Trankit[base] |
95.22 |
96.25 |
95.22 |
89.40 |
87.85 |
95.03 |
87.82 |
95.22 |
70.96 |
64.76 |
62.72 |
58.51 |
62.72 |
|
Stanza |
87.25 |
93.15 |
87.25 |
79.50 |
77.90 |
87.02 |
77.87 |
87.20 |
53.63 |
48.16 |
44.88 |
42.17 |
44.85 |
Performance for Stanza, UDPipe, and spaCy is obtained using their public pretrained models. The overall performance for Trankit and Stanza is computed as the macro-averaged F1 over 90 treebanks using the official evaluation script of the CoNLL 2018 Shared Task.
Named Entity Recognition
Performance comparison between Trankit large, Trankit base, and Stanza v1.1.1 on the test sets of 11 public NER datasets. Performance is based on entity micro-averaged F1.
Language |
Corpus |
Trankit[large] |
Trankit[base] |
Stanza v1.1.1 |
Arabic |
AQMAR |
76.1 |
74.8 |
74.3 |
Chinese |
OntoNotes |
80.5 |
80 |
79.2 |
Dutch |
CoNLL02 |
93.8 |
91.8 |
89.2 |
WikiNER |
95.0 |
94.8 |
94.8 |
|
English |
CoNLL03 |
92.5 |
92.1 |
92.1 |
OntoNotes |
90.2 |
89.6 |
88.8 |
|
French |
WikiNER |
92.9 |
92.3 |
92.9 |
German |
CoNLL03 |
85.3 |
84.6 |
81.9 |
GermEval14 |
89.4 |
86.9 |
85.2 |
|
Russian |
WikiNER |
93.2 |
92.8 |
92.9 |
Spanish |
CoNLL02 |
89.2 |
88.9 |
88.1 |