Implementing i18n support for Grafana dashboards

Introduction

As @wtf is currently working on i18n support for Grafana dashboards, the generic grafana-metadata-api might also help him along for properly addressing multiple JSON files for different languages through a single virtual datasource.

Example implementation / Blueprint

Provisionierung

Für Deine Arbeiten habe ich Dir auf weather nun testweise die Datei

/var/lib/grafana-metadata-api/i18n/test.en.json

hinterlegt.

HTTP Zugriff

Der Zugriff darauf klappt konkret folgendermaßen:

# GET request with filename in URI
http https://weather.hiveeyes.org/metadata/i18n/test.en.json

# POST with filename as target
echo '{"target": "i18n/test.en.json"}' | http POST https://weather.hiveeyes.org/metadata/search

Grafana Zugriff

Wie man das dynamisch und variablengesteuert à la

image

von der Datenquelle bezieht, ist unter grafana-metadata-api: Howto: Publish and consume your own JSON metadata file beschrieben. Vielleicht hilft es Dir weiter.

The “i18next” JSON translation file format

Thoughts

When thinking of i18n and Javascript, we might consider to use the JSON file layout of the translation files as defined by the i18next internationalization-framework written in and for JavaScript.

Benefits

i18next gives us access to the whole GNU gettext ecosystem including web-based collaborative translation tools and platforms like

See also i18next » Introduction » Ecosystem.

JSON Format i18next JSON v3

In fact, it’s really just key => value ;], see JSON Format - i18next documentation.

Resources


Example from the wild

Research

Grafana i18n

This is about i18n support for Grafana itself.

Grafana metadata mapping

These fine discussions and projects dedicate themselves to convenient metadata mapping through JSON files over HTTP and beyond.

Discussions

Duplicates

JSON data sources

1. simple-json-datasource

2. grafana-json-datasource (successor)

Backends

Others

Wishlist / Feature requests / Requirements

  • Address multiple JSON resources (e.g. i18next files) through single virtual datasource.
  • Perform simple key => value lookup and return value.
  • Value should be usable from a template variable assignment using a convenient query syntax.

GrafanaSimpleJsonValueMapper

Just trying the GrafanaSimpleJsonValueMapper on eltiempo.

Install

git clone https://github.com/CymaticLabs/GrafanaSimpleJsonValueMapper /opt/GrafanaSimpleJsonValueMapper
cd /opt/GrafanaSimpleJsonValueMapper
yarn install

Configure

cat /opt/GrafanaSimpleJsonValueMapper/server/data.json
{
  "en": {
    "close": "close",
    "mean": "mean"
  },
  "fr": {
    "close": "Fermer",
    "mean": "Moyenne"
  },
  "nl": {
    "close": "sluiten",
    "mean": "gemiddelde"
  }
}

Start

cd /opt/GrafanaSimpleJsonValueMapper
yarn start

Example

Dashboard

amo-i18n-tests

Add SimpleJson datasource

image

Add variable

List of variables

image

Works

image


Bottom line

Re. Wishlist / Feature requests / Requirements

Address multiple JSON resources (e.g. i18next files) through single virtual datasource.

Not there yet. Let’s just pack all into a single file as outlined above then.

Perform simple key => value lookup and return value.

Lookup expression like {"data": "$lang", "id": "mean"} feels reasonably simple.

Value should be usable from a template variable assignment using a convenient query syntax.

Looks like this works.

Outlook

We recognize

  • It’s still a hassle to enumerate all translation strings as Grafana variables. With the current implementation, it will be tedious to maintain. Sigh.

  • It’s not yet possible to directly use multiple i18next translation files as outlined above.

1 Like

Problem

The variables in text panels and metric aliases do not refresh at all when toggling dependent ones.

Investigations

While it doesn’t work with the Simple JSON datasource, it does work with InfluxDB.

Placeholder for @wtf’s way of storing this into and querying it from InfluxDB.

Backlog

A bunch of terms “reference codes” and its translations are transformed in influxDB’s lineprotocol like this (everything is a tag, the field has a dummy-value):

term,ref=close,en=close,nl=sluiten,fr=Fermer,de=schließen v=1
term,ref=mean,en=mean,nl=gemiddelde,fr=Moyenne,de=Mittel v=1
term,ref=selected,en=selected,nl=geselecteerd,fr=Sélectionné,de=ausgewählt(e) v=1
term,ref=hourly,en=hourly,nl=per\ uur,fr=par\ heure,de=stündlich v=1
term,ref=daily,en=daily,nl=per\ dag,fr=par\ jour,de=täglich v=1
term,ref=station,en=station,nl=station,fr=station,de=Station v=1

and stored in an influx database, here named vmm_i18n:
$ curl -i -XPOST 'http://localhost:8086/write?db=vmm_i18n' --data-binary @linefeed.txt

and then added as a datasource to Grafana. In this example dashboard there is first a variable named $lang of the type “Custom” with a list of the language-codes available:

en,fr,nl,de

Then for every term used a variable will be created which calls the term-code (here: particulateMatter) together with the currently selected value of $lang:

SHOW TAG VALUES FROM term WITH KEY IN ("$lang") WHERE "ref" = 'particulateMatter'.

This variable is now named $t_particulateMatter and can be used in every corner of your dashboard (panel headings, html-texts, graph-aliases, …) and changes automagically to another language when another is selected for variable $lang. This can be done in the variable’s drop-down – or within the paramater var-lang=xy in the dashboards’s URL.

See the test-dashboard directly starting in english, dutch, french or german and don’t forget to click within the languge-dropdown.

See two embeds of one and the same panel (showing the vocabulary) - in two languages:

While just changing the paramater in the URL used in “src” of the iframe:

/grafana/d-solo/Wr3h02gZz/wtf-i18n-tests?orgId=1&var-lang=fr&panelId=2

to:

/grafana/d-solo/Wr3h02gZz/wtf-i18n-tests?orgId=1&var-lang=nl&panelId=2

2 Likes

Hey,

very useful page.
I have been fiddling with setup and found another dashboard - i18n translations with JSON datasource

To my understanding it is built on JSON datasource setup, however fails to translate.
As far as I can tell, grafana (at least version 5.xx) uses ‘variable.value’ property most of the time. For titles or dropdowns it renders as ‘variable.text’. Having said that, after tweaking back-end for JSON datasource (experimenting with https://github.com/CymaticLabs/GrafanaSimpleJsonValueMapper.git, Michael Everett), managed to have translations on Text panel.

// file: routes/index.js:128
-                            results.push({ text: values[id], value: id });
+                            results.push({ text: values[id], value: values[id] });
// similar on other returns ...

Maybe @Andreas can say something about the JSON-approach.

The solution I posted uses just an InfluxDB for storing & calling the translations-vocabulary. But Grafana can for sure call other datasources within the template-variables.

Dear @zavyalov,

Thanks. Kudos to @wtf for that!

image

Yeah, this is failing. Thanks for noticing us. Maybe @wtf can do something about it?

Interesting. I will be happy to look into the details of your approach (Demo, Repository, Setup guide) in order to compile a list of alternatives on how to approach i18n within Grafana. Feel free to write down guidelines within this topic. It’s always nice to have people sharing a common interest together in one spot.

Feel free to elaborate further. Did I miss something? I currently can’t tell much about the i18n thing at all from a hands-on perspective but will be happy to get into the details if I can help you in any way.

With kind regards,
Andreas.

Hi gents,

Happy to share what have learned from your project and how it worked for me.

My findings mostly refer to this item (quote above) in your todo / investigation list.

In addition ‘grafana-metadata-api’ looks interesting approach to translations as lua sits within nginx and can do translations on the fly, right?
No need to setup additional server to proxy grafana requests (otherwise, something similar to GitHub - cinek810/snow-grafana-proxy: Connector for grafana simple-json data source and ServiceNow incidents retieved over ServiceNow API is needed, IMO).

— OK, it is for a limited set of use cases. Don’t know how to intervene results the grafana got from other datasources, for example, results are from postgres and the translation files are hosted separately.
If there is something I have missed on how grafana processes data, drop link, happy to dig into it.

Best
Vasiliy

1 Like

No, @Andreas, that is your child! ;)

I really wasn’t in touch with the JSON-approach.

I see. Sorry that I’ve almost forgot what I did there. @zavyalov: Good to see this was helpful to you. I actually abandoned this approach after seeing @wtf was successful with his approach.

2 Likes

Just FYI.

I tried solution with GrafanaSimpleJsonValueMapper and it works but not for aliases in visualizations.
I read the comments that solution presented by @wtf at vmm_i18n can work with aliases.
I have some questions. The server ‘localhost:8086’ is the same server as your project data connection? So it is just about placing a file “vmm_i18n” on you InfluxDB server and later use it when creating variables. Which tool do you use to place this file on a InfluxDB server? Curl? Do you know if it can be done in Windows? If placing a file ‘vmm_i18n’ by curl command is enough to make it readable by Grafana?

1 Like

I will reply to myself. I found out that in Influx DB it is possible to upload a file in line protocol format in admin panel in InfluxDB. Go to Load data → Buckets → Add data (button on specific bucket).
To be able to use InfluxQL in Grafana 8.x you need to link your bucket to database by command.
Query data with InfluxQL | InfluxDB OSS 2.0 Documentation (influxdata.com)
I hope it will help anyone.

3 Likes