Add a language
By default pii-airlock looks for names in English and French. It checks every configured language on every text and combines the results, so mixed text is handled.
Add German as an example
Create
.pii-airlock.yamlin your project:yamllanguages: [en, fr, de] models: de: de_core_news_lgDownload the models and check:
bashpii-airlock download-models pii-airlock doctordownload-modelspicks up the new language from your file. If it reports that your environment has nopip(common with pipx), it prints the exactpipx injectcommands to run instead.
More models: spaCy models list.
Many languages with one model (GLiNER)
If you work in many languages, one multilingual model can replace one model per language.
bash
pipx inject pii-airlock 'pii-airlock[gliner]'yaml
ner_backend: glinerbash
pii-airlock download-models| Standard spaCy models | GLiNER | |
|---|---|---|
| Download size | small | about 1.1 GB, plus PyTorch |
| Speed per message | about 6 ms | about 40 ms |
| Names found correctly | good | better, with far fewer false alarms |
Emails, phone numbers, IDs and secrets are found by patterns in both modes.