All You Wanted To Know About HERE Documents in Linux

By | 10/06/2013

HERE document is a special purpose code block through which text inputs can be supplied to programs on command line and within shell scripts. This way, HERE documents provide a good alternative for fetching inputs from text files.

In this article, we will understand most of the important aspects of HERE documents through some practical examples. Also, towards the end of article will be a list of resources which are worth read on various practical aspects of HERE documents.

 

HERE documents in Linux

You can think of HERE documents as a token centric way of providing multi-line input to programs on command line as well as within shell scripts. Though the standard token used is HERE but there is not restriction on token name. Some would also like to call it as a delimiter based approach rather than a token based as it sounds more relevant.

NOTE – HERE documents are not Linux specific but this article is centred around Linux and shell scripts used are bash shell scripts.

Here are 8 examples that would give you a good introduction on most of the practical aspects of HERE documents.

1. A basic example – Easily get text inputs through HERE documents

$ cat << HERE
> Hi
> How are you
> HERE
Hi
How are you

So you can see that the token HERE acts as a delimiter for cat command while reading multi-line input.

It’s not necessary that you always use the keyword HERE. Some other relevant delimiter can be used.

$ cat << EOF
> Hello
> How have you been
> EOF
Hello
How have you been

You can use any other delimiter but make sure that it is not a keyword or something that you end up writing as part of your text. The input will be terminated as soon as the delimiter is encountered.

2. Manipulating environment variables

Environment variables are expanded by default while using HERE documents.

Here is an example :

$ cat << EOF
> Home directory is $HOME
> EOF
Home directory is /home/mylinuxbook

So we see that the environment variable HOME was replaced with its value.

While using HERE documents, there could be situations where you would want the environment variable to not get replaced with its value. This can be done by enclosing delimiter with single quotes.

Here is an example :

$ cat << 'EOF'
> Home directory can be obtained by running echo $HOME
> EOF
Home directory can be obtained by running echo $HOME

So we see that the environment variable HOME was not replaced with its values.

3. Manipulating leading TABS

If you have leading TABS in the content you are using then HERE documents honours TABS by default.

Here is an example :

#!/bin/bash
cat << EOF
        Hi
        This is an example
        of text with TABS.
EOF

Lets name the shell script as HEREtabs.sh. Now, lets run the shell script.

Here is the output :

$ ./HEREtabs.sh 
    Hi
    This is an example
    of text with TABS.

So we see that the TABS were present in the output.

Though HERE documents honour leading TABS by default, but they also provide a way through which these leading tabs can be removed in output. This can be done by using a ‘-’ sign after <<.

Here is an example :

#!/bin/bash
cat <<- EOF
        Hi
        This is an example
        of text with TABS.
        EOF

Here is the output when above shell script was run :

$ ./HEREtabs.sh 
Hi
This is an example
of text with TABS.

So we see that initial TABS were removed in the output.

Whenever you want your script to be well indented but do not want the leading TABS to be processed by HERE documents. You can use this trick. :-)

NOTE- Only leading TABS are affected by using <<- but not the white spaces.

4. What’s the deal with with warnings of type “here-document at line x delimited by end-of-file“?

While working on the examples in point 3 above, I frequently encountered warnings like :

warning: here-document at line 2 delimited by end-of-file (wanted `EOF’)

So, I thought to share my understanding on these warnings and how I resolved them.

Consider the following shell script :

#!/bin/bash
cat << EOF
        Hi
        This is an example
        of text with TABS.
        EOF

Here is the output of script shown above :

$ ./HEREtabs.sh 
./HEREtabs.sh: line 6: warning: here-document at line 2 delimited by end-of-file (wanted `EOF')
    Hi
    This is an example
    of text with TABS.
    EOF

So, if you observe the warning (highlighted in bold above), it says that it was not able to find out the delimiter EOF to delimit the text passed to cat command. But, as you can see that we have passed EOF towards the end of the script then why it complained? Also, in the output, the delimiter was also displayed.

Well, it complained because of the leading TAB before EOF. As you can see that no ‘-’ is being used, this means that leading TABS will be honoured in this case. But, in this case, HERE documents expect the delimiter (EOF in our case) to be present without leading TAB.

So, if we remove the leading TAB, the warning will go away.

Similarly, when I was working on the example where ‘-’ was to be used so that TABS get removed in the output, I again faced the same problem :

#!/bin/bash
cat <<- EOF
        Hi
        This is an example
        of text with TABS.
        EOF
$ ./HEREtabs.sh 
./HEREtabs.sh: line 6: warning: here-document at line 2 delimited by end-of-file (wanted `EOF')
Hi
This is an example
of text with TABS.
EOF

This time it was a bit difficult to find as because of ‘-’, TABS were going to get removed so leading TAB was not the problem now. After a bit of research and debugging, I concluded that it was due to a stray space after the EOF in last line. Once that was removed, the warning went away.

So, it is always advisable to be cautious while typing of copy-pasting text that is to be used with HERE documents.

One thought on “All You Wanted To Know About HERE Documents in Linux

Leave a Reply

Your email address will not be published. Required fields are marked *