Bash Loops with examples

Ready to dive into Bash looping? With the popularity of Linux as a free operating system, and armed with the power of the Bash command line interface, one can go further still, coding advanced loops right from the command line, or within Bash scripts.

Harnessing this power, one can manipulate any document, any set of files, or implement advanced algorithms of almost any type and flavor. You are unlikely to run into any limitations if you use Bash as the basis for your scripting, and Bash loops form a powerful part of this.

That said, Bash loops sometimes can be tricky in terms of syntax and surrounding knowledge is paramount. Today we present with you a set of bash loop examples to help you upskill quickly and become Bash loop proficient! Let’s get started!

In this tutorial you will learn:

How Bash for, while and until based loops work, with examples
How Bash requires terminating of loop-starting statements before the do…done section of the loop can follow, and how this relates to if and other statements
How to implement basic and medium advanced bash loops
How subshells work and how they can be used inside bash loop scope declarations
How to start coding loops defensively, avoiding errors in the output
How to code one-liners (a common term used amongst Bash developers) on the command line versus implement the same code in a Bash script
How the ; syntax idiom is an important matter when it comes to coding Bash loops, both on the command line and in scripts

Bash Scripting – Bash Loops with examples

Bash Loops Examples

Let us start with a basic for loop:
```
$ for i in $(seq 1 5); do echo $i; done
1
2
3
4
5
```
As you can see, basic for loops in Bash are relatively simple to implement. Here are the steps:

for: Indicates we want to start a new for based loop
i: a variable we will be using to store the value generated by the clause inside the in keyword (namely the sequence just below)
$(seq 1 5): This is executing a command inside another sub-shell.

To understand how this works, consider this example:
```
$ seq 1 5
1
2
3
4
5
```
Basically, the $() syntax can be used whenever (and wherever!) you want to start a new subshell. This is one of the most powerfull features of the Bash shell. Consider for example:
```
$ cat test.txt
1
2
$ echo "$(cat test.txt | head -n1)"
1
```
As you can see, here the subshell executed `cat test.txt | head -n1` (`head -n1` selects only the first line) and then echo’ed the output of that subshell.

Let’s continue analyzing our for loop above:

;: This is very important. In bash, any “action”, like for example a ‘for’ loop starting, or an ‘if’ statement test, or a while loop etc. needs to be terminated with a ‘;’. Thus, the ‘;’ is here *before* the do, not after. Consider this very similar if example:
```
$ if [ "a" == "a" ]; then echo "yes!"; fi
yes!
```
Notice how again the ; is before the then, not after. Please do not let this confuse you while scripting for or while loops, if statements etc. Just remember that every action needs to be terminated before any new action, and thus for or if needs to be terminated before the next action which is ‘then’ in the if statement example, and do in the for loop above!

Finally, we have:

do: Indicating that for what comes before ... do ... what comes hereafter. Note again that this action word is after the closing ; used to close the for loop opening statement.
echo $i: Here we output the value stored into the i variable ($i)
;: Terminate the echo statement (terminate each action)
done: Indicate that this is the end of our loop
Let’s take this same example but write it differently:
```
$ for i in 1 2 3 4 5; do echo $i; done
1
2
3
4
5
```
You can see now how this relates to the example above; it is the same comment, though here we did not use a subshell to generate an input sequence for us, we manually specified it ourselves.

Does this set your head off racing about possible uses a bit? So it should 🙂 Let’s do something cool with this now.
Increasing the complexity of our for loop to include files:
```
$ ls
1.txt  2.txt  3.txt  4.txt  5.txt
```
```
$ head -n1 *.txt
==> 1.txt <==
1

==> 2.txt <==
1

==> 3.txt <==
1

==> 4.txt <==
1

==> 5.txt <==
1
```
```
$ for i in $(ls *.txt); do cat "$i" | head -n1; done
1
1
1
1
1
```
Can you work out what is happening here? Looking at the new parts of this for loop, we see:
$(ls *.txt): This will list all txt files in the current directory, and note that the name of those files will be stored in the i variable, one file per/for each loop the for loop will run through.

In other words, the first time the loop (the part between do and done) happens, $i will contain 1.txt. The next run $i will contain 2.txt and so on.

cat "$i" | head -n1: Here we take the $i variable (as we have seen this will be 1.txt, followed by 2.txt etc.) and cat that file (display it) and take the first line of the same head -n1. Thus, 5 times 1 is output, as that is the first line in all 5 files as we can see from the prior head -n1 across all .txt files.
How about a very complex one now?
```
$ tail -n1 *.txt
==> 1.txt <==
1

==> 2.txt <==
2

==> 3.txt <==
3

==> 4.txt <==
4

==> 5.txt <==
5
```
```
$ for i in $(ls *.txt 2>/dev/null); do echo -n "$(tail -n1 $i)"; echo " from $i !" ; done
1 from 1.txt !
2 from 2.txt !
3 from 3.txt !
4 from 4.txt !
5 from 5.txt !
```
Can you workout what is happening here?

Let's analyze it step-by-step.

for i in : We already know this; start a new for loop, assign variable i to whatever follows in the in clause
$(ls *.txt 2>/dev/null): The same as the command above; list all txt files, but this time with a bit of definitive error-avoiding protection in place. Look:
```
$ for i in $(ls i.do.not.exist); do echo "just testing non-existence of files"; done
ls: cannot access 'i.do.not.exist': No such file or directory
```
Not very professional output! Thus;
```
$ for i in $(ls i.do.not.exist 2>/dev/null); do echo "just testing non-existence of files"; done
```
No output is generated by this statement.

Let's continue our analysis:

; do: terminate the for loop starting statement, begin the do...done section of our loop definition
echo -n "$(tail -n1 $i)";: Firstly, the -n stands for do not output the trailing newline at the end of the requested output.

Next, we are taking the last line of each file. Note how we have optimized our code from above? i.e. instead of doing cat file.txt | tail -n1 one can simply do tail -n1 file.txt - a shorthand which new Bash developers may easily miss. In other words, here we a simply printing 1 (the last line in 1.txt) immediately followed by 2 for 2.txt etc.

As a sidenote, if we did not specify the followup echo command, the output would have simply been 12345 without any newlines:
```
$ for i in $(ls *.txt 2>/dev/null); do echo -n "$(tail -n1 $i)"; done
12345$
```
Notice how even the last newline is not present, hence the output before the prompt $ returns.

Finally we have echo " from $i !"; (showing us the from 1.txt ! output) and the closure of the loop by the done.

I trust by now you can see how powerful this is, and how much control one can exert over files, document contents and more!

Let's generate a long random string with a while loop next! Fun?

Using a while loop to generate a random string:

$ RANDOM="$(date +%s%N | cut -b14-19)"
$ COUNT=0; MYRANDOM=; while true; do COUNT=$[ ${COUNT} + 1 ]; if [ ${COUNT} -gt 10 ]; then break; fi; MYRANDOM="$MYRANDOM$(echo "${RANDOM}" | sed 's|^\(.\).*|\1|')"; done; echo "${MYRANDOM}"
6421761311

That looks complex! Let's analyze it step by step. But first, let's see how this would look inside a bash script.

Example of the same functionality, implemented inside a Bash script:
```
$ cat test.sh
#!/bin/bash

RANDOM="$(date +%s%N | cut -b14-19)"

COUNT=0
MYRANDOM=

while true; do 
  COUNT=$[ ${COUNT} + 1 ]
  if [ ${COUNT} -gt 10 ]; then 
    break
  fi
  MYRANDOM="$MYRANDOM$(echo "${RANDOM}" | sed 's|^$.$.*|\1|')"
done

echo "${MYRANDOM}"
```
```
$ chmod +x test.sh
$ ./test.sh
1111211213
$ ./test.sh 
1212213213
```
It's quite surprising at times that such complex bash looping code can so easily be moved into a 'one-liner' (a term which Bash developers use to refer to what is reality a small script but implemented directly from the command line, usually on a single (or at maximum a few) lines.

Let's now start to analyze our last two examples - which are very similar. The small differences in code, especially around the idiom ';' are explained in example 7 below:

RANDOM="$(date +%s%N | cut -b14-19)" on Line 4: This takes (using cut -b14-19) the last 6 digits of the current epoch time (The number of seconds that have passed since 1 January 1970) as reported by date +%s%N and assigns that generated string to the RANDOM variable, thereby setting a semi-random entropy to the RANDOM pool, in simple terms "making the random pool somewhat more random".

COUNT=0 on Line 6: set the COUNT variable to 0
MYRANDOM= on Line 7: set the MYRANDOM variable to 'empty' (no value assigned)
while...do...done between Line 9 and Line 15: this should be clear now; start a while loop, run the code between the do...done clauses.
true: and as long as the statement which follows the 'while' is evaluated as true, the loop will continue. Here the statement is 'true' which means that this is an indefinite loop, untill a break statement is given.
COUNT=$[ ${COUNT} + 1 ] on Line 10: Increase our COUNT variable by 1
if [ ${COUNT} -gt 10 ]; then on Line 11: An if statement to check if our variable is greater then -gt 10, and if so execute the then...fi part
break on Line 12: This will break the indefinite while loop (i.e. when COUNT is greater then 10 the loop will end)
MYRANDOM="... on Line 14: We are going to assign a new value to MYRANDOM
$MYRANDOM on Line 14: First, take what we already have inside this variable, in other words, we will be appending something at the end of what is already there, and this for each subsequent loop
$(echo "${RANDOM}" | sed 's|^$.$.*|\1|') on Line 14: This is the part which is added each time. Basically, it echo's the RANDOM variable and takes the first character of that output using a complex regular expression in sed. You can ignore that part if you like, basically it states "take the first character of the $RANDOM variable output and discard everything else"

You can thus see how the output (for example 1111211213) is generated; one character (left-to-right) at the time, using the while loop, which loops 10 times as a result of the COUNT counter variable checking.

So why is the output often in the format of 1,2,3 and less of other numbers? This is because the RANDOM variable returns a semi-random variable (based on the RANDOM=... seed) which is in the range of 0 to 32767. Thus, often this number will start with 1, 2 or 3. For example 10000-19999 will all return in 1 etc. as the first character of the output is always taken by the sed!
A short script to highlight the possibility to arrange (or style) bash looping code in a different manner without using the ; idiom.

We need to clarify the small differences of the bash script versus the one-liner command line script.

NOTE
Note that in the bash script (test.sh) there are not as many ; idioms. This is because we now have split the code over multiple lines, and a ; is not required when there is an EOL (end of line) character instead. Such a character (newline or carriage return) is not visible in most text editor's but it is self explanatory if you think about the fact that each command is on a separate line.

Also note that you could place the do clause of the while loop on the next line also, so that it becomes unnecessary to even use the ; there.
```
$ cat test2.sh 
#!/bin/bash

for i in $(seq 1 3)
do 
  echo "...looping...$i..." 
done
```
```
$ ./test2.sh 
...looping...1...
...looping...2...
...looping...3...
```
I personally much prefer the syntax style given in Example 6, as it seems clearer what the intention of the code is by writing the loop statement in full on one line (alike to other coding languages), though opinions and syntax styles differ per developer, or per developer community.
Finally, let us take a look at a Bash 'until' loop:
```
$ NR=0; until [ ${NR} -eq 5 ]; do echo "${NR}"; NR=$[ ${NR} + 1 ]; done
0
1
2
3
4
```
Let's analyze this example:

NR=0: Here set a variable named NR, to zero
until: We start our 'until' loop
[ ${NR} -eq 5 ]: This is our if condition, or better our until condition. I say if as the syntax (and the working) is similar to that of the test command, i.e. the underlaying command which is used in if statements. In Bash, the test command may also be represented by single [' '] brackets. The ${NR} -eq 5 test means; when our variable NR reaches 5, then the test will become true, in turn making the until loop end as the condition is matched (another way to read this is as 'until true' or 'until our NR variable equals 5'). Note that once NR is 5, the loop code is no longer executed, thus 4 is the last number displayed.
;: Terminate our until statement, as explained above
do: Start our action chain to be executed until the tested statement becomes true/valid
echo "$NR;": echo out the current value of our variable NR
NR=$[ ${NR} + 1 ];: Increase our variable by one. The $[' ... '] calculation method is specific to Bash
done: Terminate our action chain/loop code

As you can see, while and until loops are very similar in nature, though in fact they are opposites. While loops execute as long as something is true/valid, whereas until loops execute as long as something is 'not valid/true yet'. Often they are interchangeable by reversing the condition.

Conclusion

I trust you can start seeing the power of Bash, and especially of for, while and until Bash loops. We have only scratched the surface here, and I may be back later with further advanced examples. In the meantime, leave us a comment about how you are using Bash loops in your day-to-day tasks or scripts. Enjoy!

Bash Loops Examples

Conclusion

Related Linux Tutorials: