Process a File line by line

Process a File line by line :

There are Numerous methods to achieve the same one such method is discussed here :

Method 1:

Let’s start with the most common method that I see, which is catting a file and piping the file output to a while read loop. On each loop iteration a single line of text is read into a variable named LINE. This continuous loop will run until all of the lines in the file have been processed one at a time.

            The pipe is the key to the popularity of this method. It is intuitively obvious that the output from the previous command in the pipe is used as input to the next command in the pipe. As an example, if I execute the df command to list file system statistics and it scrolls across the screen out of view, I can use a pipe to send the output to the more command, as in the following command:

df | more

When the df command is executed, the pipe stores the output in a temporary system file. Then this temporary system file is used as input to the more command, allowing me to view the df command output one page/line at a time. Our use of piping output to a while loop works the same way; the output of the cat command is used as input to the while loop and is read into the LINE variable on each loop iteration. Look at the
complete function in Listing 2.2.

function while_read_LINE
{
cat $FILENAME | while read LINE
do
echo “$LINE”
:
done
}

Each of these test loops is created as a function so that we can time each method using the shell script. You could also use () C-type function definition if you wanted, as shown in Listing 2.3.

while_read_LINE ()
{
cat $FILENAME | while read LINE
do
echo “$LINE”
:
done
}

Whether you use the function or () technique, you get the same result. I tend to use the function method more often so that when someone edits the script they will know the block of code is a function. For beginners, the word “function” helps understanding the whole shell script a lot. The $FILENAME variable is set in the main body of the shell script. Within the while loop notice that I added the no-op (:) after the echo statement. A no-op (:) does nothing, but it always has a 0, zero, return code. I use the no-op only as a placeholder so that you can cut the function code out and paste it in one of your scripts. If you should remove the echo statement and leave the no-op, the while loop will not fail; however, the loop will not do anything either.