Summary of long answer: I suggest that a canonical question+answer about using dedicated tools to process structured documents may not be useful for avoiding duplicates because the actual issues presented in questions are often unique. Rather than pointing to a duplicate question+answer, users should be helped directly with their individual issues and referred to a Meta question that describes the limitations of standard Unix tools for processing structured documents, and possible alternatives.
There is also a tangentially related rant at the end.
The problem is that if there were one such canonical question+answer, it would be the duplicate of very few questions, as the only thing that would be "duplicated" is the explanation of why it may be better to use dedicated tools to process structured documents. The actual issue that any one question may present would likely not be a duplicate of that canonical question and answer.
It would be the Why is Kali Linux so hard to set up? Why won't people help me? question all over again, i.e. a question that can't be used as a duplicate as its answer doesn't answer any real-life questions. Instead, we should try to help those people, or close their questions as unclear or as dupes of other real questions, whichever is more appropriate. When answering these questions, we may use the potentially misguided choice of tools, making sure the restrictions, conditions, and potential issues are clear, or we could do as I usually do and ignore the user's pick of tools. Or you do both of the above, which will become tedious in the long run if you tend to answer many of these types of questions.
Regarding the issue with Kali Linux, the "Why won't people help me?" question still serves a purpose. Not as a question that we close dupes against, but as a reference that we can point to when the topic of why it may be a good idea to try another Linux distribution in place of Kali arises (because a user is either in too deep water with Kali or are trying to use the distribution to do things it clearly not designed to do). We still help the user, but we can also provide a hint that they may want to consider another platform for their work, and why. Such a hint can be provided by linking the Kali question on our Meta site.
So, (and I think this would be a really good idea) we could potentially have a Meta question asking, "Why am I told not to parse structured document formats with line-oriented tools?" (or "Why won't people help me sed
-uce my JSON docs?" something even more awk
-wardly catchy), with appropriate answers that describe the basic issues with using the standard Unix line-based text processing tools, when these tools are perfectly fine to use (because they are sometimes the correct choice if we know enough about the input), and what the alternatives are, etc.
Tangent, no longer really an answer to the Meta-question:
One of the issues I personally have with text-processing answers in general, is that they are sometimes written as one-off answers, i.e., "Run this one-liner and it will do what you need!". Even though answers like these may answer the specific question in the form that it happened to be posed, we have no clue where the code will be used or what changes will be retro-fitted to it, or what input some random future user will try to pass through it may look like.
It is, therefore, vital that we point out the limitations on answers that try to parse e.g. YAML or XML with tools like grep
, sed
, and awk
, as a way of educating users about the fragility and proper use of these standard Unix tools.
I don't want any medical practitioner to use journal systems based on parsing XML with awk
, or a car manufacturer filtering JSON equipment test data with sed
, just because some lazy programmer lifted a one-liner throwaway solution from a "peer reviewed" Unix site.
Users who don't think the code in their answers here on U&L will be used in production systems must think again.
awk
,sed
andgrep
is not going to end up being a good investment of anyone's time, please read [[ this answer ]]. Now, regarding your question,mlr
has the--do-what-I-want
flag…"