Interested in Contributing?

Scored Systems

System Submitter System Notes Constraint Run Notes BLEU BLEU-cased TER BEER 2.0 CharactTER
sharpL  (Details) sharp
sharp
yes trained data from wmt2019

50.3

49.7

0.398

0.704

0.356

Microsoft-Marian  (Details) marcinjd
Microsoft
Marian, Transformer-big ensemble x4. With filtered, clean, and domain-weighted paracrawl. Also domain-weighthing original parallel data. Decoder-time ensemble with in-domain Transformer-LM. Right-to-left scoring with Transformer-big models. yes

48.9

48.3

0.407

0.697

0.362

NMT-SMT Hybrid  (Details) fstahlberg
University of Cambridge
MBR-based combination of neural models and SMT yes

47.1

46.6

0.415

0.691

0.369

NTT Transformer-based System  (Details) makoto-mr
NTT
Based on Transformer Big model. Trained with filtered version of CommonCrawl, ParaCrawl and synthetic corpus of newscrawl2017. R2L reranking. yes

47.0

46.5

0.426

0.688

0.370

KIT Primary Submission  (Details) pianist
Karlsruhe Institute of Technology
Primary Submission yes

46.9

46.3

0.428

0.687

0.382

MMT production system  (Details) nicolabertoldi
MMT srl
Transformer-based neural MT; single model; single pass decoding. no Trained on public and proprietary data.

46.7

46.2

0.432

0.682

0.387

Facebook-FAIR  (Details) edunov
Facebook FAIR
Ensemble of six self-attentional models with back-translation data according to https://arxiv.org/abs/1808.09381 yes

46.5

46.1

0.423

0.689

0.381

Ubiqus-NMT  (Details) vince62s
Ubiqus
Base transformer Include a selection of Paracrawl Include WMT16 Rico's BackT yes

46.1

45.6

0.422

0.685

0.383

Contrastive (Single)  (Details) pianist
Karlsruhe Institute of Technology
Single Transformer (Base) Model yes

45.7

45.1

0.435

0.682

0.389

uedin-en-de-single-transfomer  (Details) ugermann
University of Edinburgh
Single transformer trained on WMT2017 data plus a selection of paracrawl. yes

44.9

44.4

0.441

0.676

0.391

MMT contrastive 2  (Details) nicolabertoldi
MMT srl
Transformer-based neural MT; single model, single pass decoding. yes Trained on a filtered version of the supplied data.

44.6

44.2

failed

0.673

0.399

JHU  (Details) jhu-nmt
Johns Hopkins University
Marian Deep RNN yes Contrastive run, fine-tuned to previous test sets (but not R2L reranking)

43.9

43.4

0.448

0.672

0.391

JHU  (Details) jhu-nmt
Johns Hopkins University
Marian Deep RNN yes Marian deep model, ensemble of 4 runs using base data (without Paracrawl), re-back-translated news 2016. R2L Reranking. Primary.

44.0

43.4

0.449

0.672

0.392

uedin-en-de-single-transformer-reranked  (Details) ugermann
University of Edinburgh
Single transformer, reranked with two R2L transformers. yes

43.8

43.2

0.450

0.669

0.397

JHU  (Details) jhu-nmt
Johns Hopkins University
Marian Deep RNN yes Marian deep model, ensemble of 4 runs using base data (without Paracrawl), re-back-translated news 2016. Not final system yet.

43.6

43.0

0.453

0.670

0.394

MMT contrastive  (Details) nicolabertoldi
MMT srl
Transformer-based neural MT; single model, single pass decoding. yes Trained on a filtered version of the supplied data. German de-compounding applied.

42.9

42.5

0.463

0.667

0.411

uedin-en-de-2+2-transformer  (Details) ugermann
University of Edinburgh
2 transformers ensembled, reranked with 2 R2L systems. Include paracrawl yes

42.3

41.8

0.463

0.663

0.405

LMU-nmt-reranked-wmt18-en-de  (Details) Matthias.Huck
LMU Munich
Nematus encoder-decoder NMT (single model + r2l reranking), like last year yes

40.6

40.0

0.480

0.655

0.421

NJUNMT  (Details) ZhaoChengqi
Nanjing University
transformer base without back translation yes transformer base without back translation

40.6

40.0

0.496

0.647

0.436

LMU-nmt-single-wmt18-en-de  (Details) Matthias.Huck
LMU Munich
Nematus encoder-decoder NMT (single model), like last year yes

39.3

38.8

0.492

0.647

0.433

parfda  (Details) bicici
yes en-de using PRO for tuning

27.3

26.7

0.620

0.591

0.570

Wink  (Details) anything
uni saarland
Out-of-domain data no

22.3

21.9

0.706

0.540

0.741

LMU-unsupervised-nmt-wmt18-en-de  (Details) Matthias.Huck
LMU Munich
Unsupervised NMT (no parallel training corpora) yes

15.8

15.5

0.762

0.500

failed

RWTH Unsupervised NMT Ensemble  (Details) yunsukim
RWTH Aachen University
(Unsupervised) Transformer with shared encoder/decoder, separate top-50k word vocabs, iterative back-translation, ensemble 4x yes

15.9

14.8

0.753

0.514

0.607

RWTH Unsupervised NMT Single  (Details) yunsukim
RWTH Aachen University
(Unsupervised) Transformer with shared encoder/decoder, separate top-50k word vocabs, iterative back-translation yes

15.6

14.5

0.758

0.510

0.615

LMU-unsupervised-pbt-wmt18-en-de  (Details) Matthias.Huck
LMU Munich
Unsupervised (no parallel training corpora), BWEs + PBT yes

14.6

14.3

0.791

0.518

0.627

Ubiqus-NMT  (Details) vince62s
Ubiqus
Base transformer Include a selection of Paracrawl Include WMT16 Rico's BackT yes