Language normalisation: check languageOfMaterial / scriptOfMaterial fields for validity...

[mikesname]

Backend doesn't enforce this I don't think.

Normalise anything that's dodgy...

[mikesname]

@herminiogg this is, as anticipated, a bit messy. The backend does a best effort to normalise values to ISO639-2 but stores the original if that fails, rather than throwing an error. It's liable to fail normalisation especially if the language names are not in English.

I will try and clean up what I can, but I think we'll have to be a bit tolerant on LOD conversion too (ignoring invalid codes), unless we make the backend much stricter.

[herminiogg]

Yes, the problem in the LOD conversion is that we are a bit too tolerant so everything is converted as it is and we end up with things like https://lod.ehri-project-test.eu/languages/English.html, https://lod.ehri-project-test.eu/languages/eng.html and https://lod.ehri-project-test.eu/languages/Anglais.html which should be all the same entity. Another possibility would be to post-process this and build an equivalences table which could potentially be used later in the EHRI Portal.