-
Notifications
You must be signed in to change notification settings - Fork 1
Standardization does not preserve metadata of nested fields #56
Copy link
Copy link
Open
Labels
bugSomething isn't workingSomething isn't working
Description
Describe the bug
Standardization keeps metadata of root fields, but does not preserve metadata of nested fields.
To Reproduce
Run standardization of a dataframe with a following schema:
{
"type": "struct",
"fields": [
{
"name": "STRING_FIELD",
"type": "string",
"nullable": true,
"metadata": {
"maxLength": 8
}
},
{
"name": "NESTED_STRUCT",
"type": {
"type": "struct",
"fields": [
{
"name": "NESTED_STRING",
"type": "string",
"nullable": true,
"metadata": {
"maxLength": 5
}
}
]
}
}
]
}The result will look like this (nested field's metadata is lost):
{
"type": "struct",
"fields": [
{
"name": "STRING_FIELD",
"type": "string",
"nullable": true,
"metadata": {
"maxLength": 8
}
},
{
"name": "NESTED_STRUCT",
"type": {
"type": "struct",
"fields": [
{
"name": "NESTED_STRING",
"type": "string",
"nullable": true,
"metadata": { }
}
]
}
}
]
}STRING_FIELDhas metadata retained.NESTED_STRINGhas list metadata field"maxLength": 5
Expected behavior
Standardization should preserve all metadata.
Screenshots
--
Desktop (please complete the following information):
--
Additional context
--
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working