ls -ltr on the terminal lists the files but says "No such file or directory" in a shell script

Question

I am new to Bash and am practicing some shell scripts. I wanted to write a script to list all the folders and their file count, if a root path is passed as a parameter value from the terminal. The problem I am facing is when I run the specific command on the terminal using ls or find it works.

Desired output:

/Users/abc/Documents : 10
/Users/abc/Documents/Subfolder1 : 11
/Users/abc/Documents/Subfolder2 : 12
...

The code is given below:

#Assign path passed from terminal to var
ROOT_PATH=$1

#list all dirs under the root path to traverse
find "$ROOT_PATH" -type d -name "*" | grep -v ".git" | while read line; do echo \"$line\"; done > ./Temp.txt 

while IFS= read -r line
do
        echo "$line"
        let a=0
        find "$line" -type f -name "*"  #| wc -l <<<a | echo "$line :  $((a-1))"
        echo "---Next---"
done < ./Temp.txt

I execute the script as bash script.sh /Users/abc/Documents/ and get the error find: "/Users/abc/Documents": No such file or directory

The Temp.txt file contains:

cat Temp.txt 
"/Users/abc/Documents/"
"/Users/abc/Documents/Subfolder1"
"/Users/abc/Documents/Subfolder2"

The problem is, if I just pick up the find command from the while loop and run it on the terminal (I have a mac), it works!!

find "/Users/abc/Documents/" -type f -name "*"
/Users/abc/Documents///1.txt
/Users/abc/Documents///2.txt
/Users/abc/Documents///3.txt

What am I doing wrong? I've read elsewhere on globs and to quote all variables in the script which I am doing. I am also ensuring to store full paths and not expandable ones in the Temp.txt file. The same problem also happens with the ls command in the while loop.

Any help will be greatly appreciated!!

Answer 1

What am I doing wrong?

Several things that are rather common bugs when you are "new to Bash".

In echo \"$line\" the double-quotes are escaped, not special, they get to the output and $line is unquoted, which is by itself wrong. You actually see the quotes in the file. Then they get stored in $line (at the second read). And then this happens: quotes in variables are not special.

A basic fix is to replace echo \"$line\" with echo "$line". This won't make the script good and robust, but at least it will work in most cases.

There is more to improve:

• There should be a shebang.

• -name "*" in find is a test that always succeeds.

• Do not use echo to print uncontrolled data.

• Use IFS= read -r line also with the first read. Or better get rid of this read because…

• Reading lines and storing in a file, only to read lines from the file to process further seems over-complicated. If you really want to process pathnames this way, pipe from grep directly to the second while.

• You don't even need this grep. Append ! \( -name '*.git*' -prune \) to your first find and get rid of grep.

• This line:

  find "$line" -type f -name "*"  #| wc -l <<<a | echo "$line :  $((a-1))"
  

  contains commented code that looks like a remnant of some voodoo scripting. I guess at some point this was not a comment, just weird code. In particular:

  • wc -l <<<a reads the here string (literally a) from its stdin and there is no point in piping anything to it, the here string will "win".
  • There is no point in piping to echo, it does not read its stdin anyway.

Some ideas about counting files: here.