A few of us were talking about this the other day in chat, as we are now quite a bit larger and with several thousand Q&As, there are certain questions that are asked very often. Things like redirecting output, process substitution, batch renaming of files, fixing/reinstalling grub, etc.

Let's compile a list of the best/most complete answers to such common questions here, and then we can always close new questions as duplicates of those. That way, we can have a single, canonical answer to these things which we can edit and make as good as we can and avoid having the dupes pointing at each other all over the place. The inspiration comes from this meta AU question:

Ideally, I would like to see an answer below that collects as many as possible. I will make a start but please edit to add more. As part of this initiative, we should also consider asking such questions and answering them ourselves. For example, I just read this question over on SU, having that kind of thing here seems like a very good idea (not specifically for hardware, although that would be useful, but that type of canonical question*).

So, the idea would be to hash it out here, and once we have a decent list, make a meta Q&A like the ones above, listing them. Then, we make sure to close as dupes of the canonical.

--

Please vote for the answers below, upvote them if you consider the suggested Q&A to be a good candidate for the canonical one and downvote them if you don't. Upvoting the question itself is nice but less useful.


* I made a start here: How can I replace a string in a file(s)?

  • 1
    yes That's a very good idea – Kiwy Feb 6 '14 at 11:16
  • I like the idea, and just seen the first one, Thought I'd give some feedback. Having all the answers combined into one answer makes it difficult to provide comments, vote, and contribute new sub-answers to a particular sub question. Maybe separate answers would work better ? One of the interesting (and valuable) aspects of this site is the way it allows different users to provide answers and the voting system. Since this is not actually a wiki, I'm now running out of chars in this comment, will not be able to edit in 5 mins etc. – X Tian Feb 9 '14 at 16:59
  • @XTian my idea was to have one, all encompassing answer to close duplicates with. It's a community wiki answer so I get no rep from it and upvotes just indicate that it works. Since to the best of my knowledge, all the solutions mentioned in the replace strings Q work, I'm not sure if it would be better to have multiple answers. It might, I'm just not sure. Why don't you try making a similar canonical Q&A on another subject and post multiple As? I think that's what Faheem is doing here. – terdon Feb 9 '14 at 17:02
  • Well I've had a go with this question, wanted to choose a problem that had solutions that could be based on multiple technologies. It was funny, someone posted a perl solution whilst I was adding answers. But noticed their solution is not part of the community wiki ! Any way to ensure this ? It does allow popular answers to percolate up and comments per solution. – X Tian Feb 9 '14 at 20:21
  • Cool @XTian, thanks! Don't worry about the CW, I just did that cause I didn't think it was fair to advertise my answer here and then get loads of rep from it if it becomes the canonical. Making posts CW in this context means that you're inviting people to edit them and that you politely decline the rep. If another user posts an answer, let them. – terdon Feb 9 '14 at 20:33
  • Actually, @XTian there is already a pretty good Q&A on that here (the one mentioned in my answer below). Most of the solutions you mention in your answer are already there and it is a nice, old and established post. I think it would be better to add any of your answers that are missing from it to that question instead of starting a whole new one. – terdon Feb 9 '14 at 20:43
  • Judging from your and other comments directly on the post, maybe it's not a good idea to duplicate the question again. So ' ... compile a list ..' means just that, a list of canonical questions. Stackoverflow doesn't have lists like this (afaik), so only way to achieve this is to write a new question, which is where I came in. The answers, if we could write such a question, would simply be references to popular q+a's that are already in existence and maybe it would be these links that are voted up/down. – X Tian Feb 10 '14 at 0:38
  • @XTian this particular question happens to be already answered quite well in the post I link to below. We tend to avoid creating duplicated info (that is actually the whole point of this thread). It's a shame because the Q&A you posted is indeed very good (and appreciated) but since the info already exists it seems pointless to duplicate. Perhaps you could post your answer to the other post, adding anything not already mentioned there? – terdon Feb 10 '14 at 15:16
  • ##How do I solve an APT dependency issue? unix.stackexchange.com/q/121180/4671 – Faheem Mitha Jul 16 '14 at 16:30
  • 6
    The best place to list canonical QAs is not as a mess on meta, but in the tag wiki for the relevant tag(s). – Gilles Sep 4 '14 at 13:35
  • The bash tag on Stack Overflow has a tag wiki with a collection of common questions. Not all of them are "canonical" in the strict sense, but the list there works well to find a quick dupe for many common problems. – tripleee Apr 25 '17 at 7:14
  • Single-quoting vs Double-quoting: unix.stackexchange.com/questions/7574/… – Jeff Schaller Jul 27 '17 at 21:38
  • this needs a good canonical answer for why you can't change a variable in a subshell (e.g. pipe into a while loop). I have to search for one every time it gets duped. – cas Jan 18 at 11:35
  • @cas do you have a suggestion? I know I've seen one but I can't find it now. – terdon Jan 18 at 12:09
  • I just used this unix.stackexchange.com/questions/9954/… because of Gilles' explanation, but I know i've seen better (more comprehensive) ones, possibly also by Gilles or maybe Stéphane – cas Jan 18 at 12:15

33 Answers 33

up vote 8 down vote accepted

Compiled list of questions

This list is compiled mostly from the answers below and some of my own, and is intended to be a quick reference for these questions.

shells

sh

bash

command line

searching filesystems

text processing

regex

history

other

Where's my disk space?

Why do du and df return different values?

We seem to generally close these Qs as dupes of this one:

Why is there a discrepancy in disk usage reported by df and du?

For example these:

I suggest we close as dupes of this one instead, I feel the answers here are better:

df says I have 20G more disk space used than du. Why?


UPDATE: @Gilles has posted The answer, so let's use that one:

Why are there so many different ways to measure disk usage?

  • 1
    agreed, but we should either improve the title of the second question (it's not very discoverable), or get Michael to merge them – strugee Feb 2 '14 at 19:42
  • @strugee that's not that big of an issue, if the others are closed as dupes of this one, the other questions will serve as signposts to it. – terdon Feb 2 '14 at 19:43
  • true. I still think it could be improved, though. – strugee Feb 2 '14 at 19:43
  • @strugee go for it, you have the rep. – terdon Feb 2 '14 at 19:45
  • way ahead of ya – strugee Feb 2 '14 at 19:45

Determining the path of an executable

Why not use "which"? What to use then?

Redirecting output

How to redirect the output of any command?

Installing/Reinstalling/Fixing grub

How can I fix/install/reinstall grub?

I just edited my answer to make it more generic but it might be a good idea to modify the title if we're going to use this one.

Batch renaming of files

Batch renaming files

  • I've added missing answers to that post. Except the generic pattern. – X Tian Feb 10 '14 at 18:35
  • @XTian that was quite useful though (very nice explanation), perhaps you could add it to your answer? – terdon Feb 10 '14 at 18:36

Where should I set environment variables?

Is there a ".bashrc" equivalent file read by all shells?

Transform/Extract info from a set of CSV files

Is there a robust command line tool for processing csv files?

An additional question/answer pair could be created. The answer(s) should include regex/sed, awk based for those simpler CSVs, as well as Perl/Python based for when simple does not work.

A few good small example CSVs in the text that clearly illustrate where simple processing will just stop.

  • 1
    The answer should include cut as well. – evilsoup Oct 12 '14 at 15:53

How do I monitor my server?

@slm's answer on this question is very good and inclusive:

Commands for determining level of usage of server

Using packages from more recent versions of a OS on earlier versions of that OS.

This question is probably most relevant on binary-based OSs like Debian/Fedora, and less relevant for source-based OSs like Gentoo/FreeBSD.

In these cases, one could give some hints about backporting, perhaps? For Debian, talk about backports, for ubuntu, PPAs, and so forth. This discussion should probably be restricted to official repositories, though.

I've posted this here: How can I install packages from a more recent version of my OS than the one I am running?.

See the output/control of a running command in another terminal

https://unix.stackexchange.com/search?q=reptyr

How does perl regexp and bash pattern work ?

This would need a basic description of different specific symbols and would need to explain the differences between the two - mode pattern vs perl regexp. This question could be linked with tag related to perl/sed/string replacement...
That's one of the most important things to master as it is used in almost any command line.

My script runs from the commandline, but not from cron

I was sure I had answered this before but could not find it or another generic answer (but I probably overlooked this).

Not able to run shell script using cron

Searching and replacing strings in files.

There are a fair number of questions asked about this with variations, and it would be good if there was a meta answer pointing to the different variations.

Search and replacing a string on specific file extensions

Replacing string based on line number

Recursively replace string in files

Search and replace full line in recursive files

Putative general Q&A:

How can I replace a string in a file(s)?

Please add others.

Remember to quote your variables

Proposed question and answer:

Handling filenames with space and unusual characters

So many answers can be simplified and easier understood, when these edge cases are omitted. So the question Why does my shell script choke on whitespace or other special characters? provides a canonical method to handle filenames with uncommon characters. #()%^*space* *newline* [];'@:<>,.

Filenames to test your shell script against

How can I test my shell script's file-handling robustness?

  • 1
    Can you suggest an existing answer as a canonical one? – terdon Feb 9 '14 at 19:27
  • @terdon - I think this is a good one. – taliezin May 7 '15 at 12:30

Find Hardware Information of my machine from command line

How to setup passwordless access to my machine?

This answer provides some of the available options.

script to automate scp in network

How to stop an application from buffering

https://unix.stackexchange.com/search?q=stdbuf

  • Can one of these be used as the Q&A on the subject? – terdon Feb 3 '14 at 14:40
  • @terdon no defintly not – Kiwy Feb 6 '14 at 11:15

Install 32bit packages on a 64bit system

Proposed canonical:

How do I run 32-bit programs on a 64-bit Debian/Ubuntu?

Gilles' answer is excellent and distro agnostic and warlock's answer mentions multiarch.

How do I solve an APT dependency issue?

What information do I need to solve an APT dependency issue?

This one comes up fairly often. Most of the time, people have no idea what to do. See also "How do I resolve unmet dependencies?" on Ask Ubuntu. Essentially all the answers to this question are notable for totally missing the point.

Extract info from a XML/HTML file.

The answer(s) should include regex/sed, awk based for those simpler structured files and extractions, as well as Perl/Python/Ruby based for when simple does not work.

A few good small example segments in the text that clearly illustrate where simple processing will just stop.

How can I copy/paste files via the command line?

Proposed canonical:

Do not parse the output of ls

Proposed question and answer:

How can I install a custom kernel?

Configuring, compiling and installing a custom Linux kernel

[Extra text added to stop the system from converting "trivial" answers to comments]

How can I repeat the previous N commands in my shell?

Proposed canonical:

How to execute consecutive commands from history?

How can I remap the keys on my keyboard to other keys (or key combinations)?

Proposed canonicals:

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .