Vega-lite data transformation to un-nest objects
Posted By: Anonymous
The data is incoming from an elasticsearch url and has the following form :
{
"took": 44,
"timed_out": false,
"hits": {
"total": 11,
"max_score": 0,
"hits": [
{
"_index": "dataindex",
"_type": "span",
"_id": "tKVUs3kBhoeKMUMeIwCv",
"_score": 0,
"_source": {
"fieldA": 272.2,
"fieldB": 73,
"fieldX": "event 1"
}
},
{
"_index": "dataindex",
"_type": "span",
"_id": "iuVetHkBhoeKMUMe4O92",
"_score": 0,
"_source": {
"fieldA": 305.2,
"fieldB": 80,
"fieldX": "event 2"
}
},
{
"_index": "dataindex",
"_type": "span",
"_id": "Yt-QwXkBhoeKMUMex3tp",
"_score": 0,
"_source": {
"fieldA": 281.8,
"fieldB": 73,
"fieldX": "event 3"
}
}
]
}
}
I wish to make a scatter plots matrix. The data array under hits.hits
can be accesses through the format.property
config.
{
"$schema": "https://vega.github.io/schema/vega-lite/v4.json",
"repeat": {
"row": [
"_source.fieldA",
"_source.fieldB"
],
"column": [
"_source.fieldB",
"_source.fieldA"
]
},
"spec": {
"data": {
"url": "url/to/elastic/query",
"format": {"property": "hits.hits", "type": "json"}
},
"mark": "point",
"encoding": {
"x": {
"field": {"repeat": "column"},
"type": "quantitative"
},
"y": {
"field": {"repeat": "row"},
"type": "quantitative"
},
"color": {
"field": "_source.fieldX",
"type": "nominal"
},
"shape": {
"field": "_source.fieldX",
"type": "nominal"
}
}
}
}
But there is still an extra _source
level that needs to be pointed everywhere (repeat, color, shape) and moreover it appears as such in the axis and legend titles :
Is there a transformation type that would get rid of this _source
level ? So the data going to the encoding
phase could act as if the source was simply
[
{
"fieldA": 272.2,
"fieldB": 73,
"fieldX": "event 1"
},{
"fieldA": 305.2,
"fieldB": 80,
"fieldX": "event 2"
}
]
Or alternatively a way to dynamically rename the axis in a repeat matrix ?
Solution
There is a way where you have to provide the fields once and it will be out on single level instead of nested. Perform calculate
transform as done below or in editor:
{
"$schema": "https://vega.github.io/schema/vega-lite/v5.json",
"transform": [
{"calculate": "datum._source.fieldA", "as": "fieldA"},
{"calculate": "datum._source.fieldB", "as": "fieldB"},
{"calculate": "datum._source.fieldX", "as": "fieldX"}
],
"repeat": {
"row": ["fieldA", "fieldB"],
"column": ["fieldB", "fieldA"]
},
"spec": {
"data": {
"values": {
"took": 44,
"timed_out": false,
"hits": {
"total": 11,
"max_score": 0,
"hits": [
{
"_index": "dataindex",
"_type": "span",
"_id": "tKVUs3kBhoeKMUMeIwCv",
"_score": 0,
"_source": {"fieldA": 272.2, "fieldB": 73, "fieldX": "event 1"}
},
{
"_index": "dataindex",
"_type": "span",
"_id": "iuVetHkBhoeKMUMe4O92",
"_score": 0,
"_source": {"fieldA": 305.2, "fieldB": 80, "fieldX": "event 2"}
},
{
"_index": "dataindex",
"_type": "span",
"_id": "Yt-QwXkBhoeKMUMex3tp",
"_score": 0,
"_source": {"fieldA": 281.8, "fieldB": 73, "fieldX": "event 3"}
}
]
}
},
"format": {"property": "hits.hits", "type": "json"}
},
"mark": "point",
"encoding": {
"x": {"field": {"repeat": "column"}, "type": "quantitative"},
"y": {"field": {"repeat": "row"}, "type": "quantitative"},
"color": {"field": "fieldX", "type": "nominal"},
"shape": {"field": "fieldX", "type": "nominal"}
}
}
}
Answered By: Anonymous
Disclaimer: This content is shared under creative common license cc-by-sa 3.0. It is generated from StackExchange Website Network.