Neural evaluation metrics for machine translation inherit their behaviour from the multilingual encoders beneath them, yet how those encoders represent low-resource African languages is rarely examined. We evaluate four large language models on English-to-Hausa and English-to-Fongbe translation across scales from 500 to 10,000 sentences, validate automatic metrics against native-speaker judgment, and then probe the multilingual representation space that neural metrics depend on. The central finding is a representation failure: XLM-R assigns near-identical vectors to semantically different texts in both languages (within-language cosine similarity above 0.99), so BERTScore cannot separate systems of visibly different quality, and COMET produces a ranking anomaly for Fongbe. This representation collapse is a central reason embeddingbased metrics are unreliable here, rather than any surface property of the languages. We further show that metric–human rank agreement is language-dependent, that in our samples system rankings stabilise by about 2,500 sentences, and that inclusion of a language in an encoder’s pretraining set does not guarantee usable representations. We frame these results as evidence that multilingual evaluation should be checked at the representation level before it is trusted.