Json transformation using jq

Question

I want to transform json to new line delimited json. I was attempting to do it multiple times using jq in bash, but I have not been able to get even close to the final output.

INPUT:

{
    "windows124": {
        "updated": "2015-01-14",
        "attribution": [],
        "description": "",
        "notes": [],
        "alt_names": [],
        "sources": [],
        "urls": ["google.com", "google.co.uk"],
        "common_name": "test",
        "uuid": "7259334c-3218-4259-aaab-896d87507f4f"
    },
    "linux124": {
        "updated": "",
        "attribution": ["Naifdddkoscn"],
        "description": "",
        "notes": [],
        "alt_names": [],
        "sources": [],
        "urls": ["https://example.com/1.pdf", "https://example.com/1.pdf", "https://example.com/1.pdf"],
        "common_name": "121212",
        "uuid": "009db412-762d-4256-8df9-eb213be01ffd"
    },
    "wikipedia123": {
        "updated": "2018-07-31",
        "attribution": [],
        "description": "",
        "notes": [],
        "alt_names": [],
        "sources": [],
        "urls": ["https://example.com/1.pdf"],
        "common_name": "test343",
        "uuid": "4d8da0af-cfd7-4990-b211-af0e990vfda0"
    }
}

WANTED OUTPUT:

{"uuid": "7259334c-3218-4259-aaab-896d87507f4f","family": "windows124","updated": "2015-01-14","attribution": [],"description": "","notes": [],"alt_names": [],"sources": [],"urls": ["google.com", "google.co.uk"],"common_name": "test"}
{"uuid": "009db412-762d-4256-8df9-eb213be01ffd","family": "linux124", "updated": "","attribution": ["Naifdddkoscn"],"description": "","notes": [],"alt_names": [],"sources": [],"urls": ["https://example.com/1.pdf", "https://example.com/1.pdf", "https://example.com/1.pdf"],"common_name": "121212"}
{"uuid": "4d8da0af-cfd7-4990-b211-af0e990vfda0","family": "wikipedia123", "updated": "2018-07-31","attribution": [],"description": "","notes": [],"alt_names": [],"sources": [],"urls": ["https://example.com/1.pdf"],"common_name": "test343"}

What I have so far is: cat deserialize.json | jq -c '.|to_entries[]'

{"key":"windows124","value":{"updated":"2015-01-14","attribution":[],"description":"","notes":[],"alt_names":[],"sources":[],"urls":["google.com","google.co.uk"],"common_name":"test","uuid":"7259334c-3218-4259-aaab-896d87507f4f"}}
{"key":"linux124","value":{"updated":"","attribution":["Naifdddkoscn"],"description":"","notes":[],"alt_names":[],"sources":[],"urls":["https://example.com/1.pdf","https://example.com/1.pdf","https://example.com/1.pdf"],"common_name":"121212","uuid":"009db412-762d-4256-8df9-eb213be01ffd"}}
{"key":"wikipedia123","value":{"updated":"2018-07-31","attribution":[],"description":"","notes":[],"alt_names":[],"sources":[],"urls":["https://example.com/1.pdf"],"common_name":"test343","uuid":"4d8da0af-cfd7-4990-b211-af0e990vfda0"}}

You can add the -c (--compact-output) command line option if you don't want the output pretty-printed — steeldriver
– steeldriver, Commented Nov 14, 2018 at 16:26

oliv · Accepted Answer · 2018-11-15 08:05:31Z

2

You can use this jq filter:

<file jq 'to_entries|map(.value + {family:(.key)})[]'

As you found out, the to_entries function allows to get the key name in order add the property family.

So the filter only creates this family object and adds it to the content of the value given by to_entries function.

The map function does the add operation to all elements of the value array.

The final [] gets rid of the outer array.

Note the output is not order in the way you posted, but the content is same. If you want the keys to be sorted, use the option -S.

edited Nov 15, 2018 at 8:05

answered Nov 14, 2018 at 16:49

oliv

2,64511 silver badges15 bronze badges

Thanks $oliv. Nicely done. Have you solved similar issue before, or how did you figure that out? If you can, please let me know how I could replicate your steps differently than doing brute force on all options.

creed
– creed

2018-11-14 19:50:34 +00:00
Commented Nov 14, 2018 at 19:50
@creed The way to solve this type of problem is indeed to try out. Read the man pages and go to the jq website to get to know the functions and how to use it. In other words, learn the language.

oliv
– oliv

2018-11-15 07:26:58 +00:00
Commented Nov 15, 2018 at 7:26

Add a comment |

Kusalananda · Accepted Answer · 2021-07-12 06:46:18Z

jq -c 'with_entries(.value.family = .key)[]' file.json

or,

jq -c 'with_entries(.value += { family: .key })[]' file.json

This inserts a family key into each sub-element of the original top-level object, with the value of the key of that element (e.g. windows124 for the first one). It then returns the sub-element as a set, which is what you appear to want.

Output for the document given in the question:

{"updated":"2015-01-14","attribution":[],"description":"","notes":[],"alt_names":[],"sources":[],"urls":["google.com","google.co.uk"],"common_name":"test","uuid":"7259334c-3218-4259-aaab-896d87507f4f","family":"windows124"}
{"updated":"","attribution":["Naifdddkoscn"],"description":"","notes":[],"alt_names":[],"sources":[],"urls":["https://example.com/1.pdf","https://example.com/1.pdf","https://example.com/1.pdf"],"common_name":"121212","uuid":"009db412-762d-4256-8df9-eb213be01ffd","family":"linux124"}
{"updated":"2018-07-31","attribution":[],"description":"","notes":[],"alt_names":[],"sources":[],"urls":["https://example.com/1.pdf"],"common_name":"test343","uuid":"4d8da0af-cfd7-4990-b211-af0e990vfda0","family":"wikipedia123"}

Stack Exchange Network

Json transformation using jq

2 Answers 2

You must log in to answer this question.

Hot Network Questions

Json transformation using jq

2 Answers 2

You must log in to answer this question.

Related

Hot Network Questions