Linux

LOOK THROUGH EVERYTHING IN DIRECTORY FOR TERMS
####################################################

Made this article into an amazing program set/script called: termsearch

Make a terms file in the directory where you want to recursively search for words. Make sure current directory is not being searched but a folder within it, or another tree structure, so that output files do not get searched.

HOW TO BY EXAMPLE:
###################
So a good example is

So if your in the folder “folder” and you need to search for the list of terms in the file “terms” thru the directory “searchthruthisdir” then you would cd to folder and then run the command

/folder/term < – put the terms to grep with here, one line at a time
/folder/searchthruthisdir <– this folder has everything you want to search thru
cd /folder
Pick a command and edit RELDIR to look like this
RELDIR=”searchthruthisdir”
Note: can use more then one reldir just seperate them with spaces, also reldir can be an absolute path, not just a relative path
For example this will look thru current working directories’ folder1 and 2 folder recursively, and also thru all of etc : RELDIR1=”folder1 folder2 /etc”
Copy paste the command
Then run the watch script if you want to follow results, or retry using retry commands then repeat the steps

Note all results will be like this:
/folder/_all_term1
/folder/_all_term2

Some of the commands also save a
/folder/_all_GREP

Where all of the results are concat

INFO ON RESULTS
###############

Results go to _all_term

Also there is a _all_GREP file for all of the output concat on the first commands (only for single thread, none multithreaded)

There is a script to search concurrently. Also a script to watch the output.

All of these are copy pasteable scripts.

STEP ONE
########

Make terms file, each term on new line

cat terms
btrfs
mkfs.btrfs
mkfs
mdadm

STEP TWO PICK YOUR COMMAND
###########################

NOTE: the -nir, n is for line numbers, i is for case insensitivity, r is for recursive search thru every subfile/subfolders

LOOK ONE BY ONE AND OUTPUT TO SCREEN (single thread – not multithreaded)
#####################################
RELDIR1=”searchthruthisdir”; for i in `cat terms`; do echo “=======================”; date; echo “Looking for: $i”; echo “=======================”; (grep -nir $i $RELDIR1 | tee _all_$i.txt;); echo ; done | tee _all_GREP.txt

SILENT IN THE BACKGROUND LOOKER (single thread – not multithreaded)
###############################
RELDIR1=”searchthruthisdir”; (for i in `cat terms`; do echo “=======================”; date; echo “Looking for: $i”; echo “=======================”; (grep -nir $i $RELDIR1 | tee _all_$i.txt;); echo ; done > _all_GREP.txt) &

TIME THE WHOLE THING FROM ABOVE (single thread – not multithreaded)
###############################
RELDIR1=”searchthruthisdir”; (time (for i in `cat terms`; do echo “=======================”; date; echo “Looking for: $i”; echo “=======================”; (grep -nir $i $RELDIR1 | tee _all_$i.txt;); echo ; done) >& _all_GREP.txt) &

BEST METHOD (MULTI THREADED):
##########################

* RUNS EACH ONE SEPERATELY, TIMES EM ALL
* MULTITHREADED

RELDIR1=”searchthruthisdir”; for i in `cat terms`; do echo “=======================”; date; echo “Looking for: $i”; echo “=======================”; (time (grep -nir $i $RELDIR1) >& _allSS_$i.txt &); echo “———————-“;echo “Started PID: $!”;(ps awfuxx | egrep grep;); echo; done; echo “##############################”; echo “FINAL JOBS:”; (ps awfuxx | egrep grep;)

Note: can use more then one reldir just seperate them with spaces, also reldir can be an absolute path, not just a relative path
For example this will look thru current working directories’ folder1 and 2 folder recursively, and also thru all of etc : RELDIR1=”folder1 folder2 /etc”

BEST ONE (solaris):
#################
Still copy pasteable jsut had to seperate it out more (maybe in your solaris version you will not have to seperate it out as much, also had to change the ps)

RELDIR1=”searchthruthisdir”;
for i in `cat terms`;
do echo “=======================”;
date;
echo “Looking for: $i”;
echo “=======================”;
(grep -nir $i $RELDIR1 >& _allSS_$i.txt &);
echo “———————-“;
echo “Started PID: $!”;
ps ax | egrep grep;
echo;
done;
echo “##############################”;
echo “FINAL JOBS:”;
ps ax | egrep grep;
WATCH THE SAVING
#################
WITHOUT PROCESS TREE:
———————-
watch tail _all*

or

while true; do “=====`date`======”; tail _all*; sleep 1; done;

WITH PROCESS TREE:
——————-
watch ‘echo “PROCESSES:”; echo “===========”; (ps awfuxx | egrep grep;); echo; echo “RESULTS:”; echo “===========”; tail _all*;’

or

while true; do clear; “=====`date`======”; echo “PROCESSES:”; echo “===========”; (ps awfuxx | egrep grep;); echo; echo “RESULTS:”; echo “===========”; tail _all*; sleep 1; done;

SHOWS NUMBER OF LINES PER FIND:
——————————-

watch ‘echo “PROCESSES:”; echo “===========”; (ps awfuxx | egrep grep;); echo; echo “RESULTS (number of lines):”; echo “==========================”; wc -l _allS*;’

while true; do clear; “=====`date`======”; echo “PROCESSES:”; echo “===========”; (ps awfuxx | egrep grep;); echo; echo “RESULTS (number of lines):”; echo “==========================”; wc -l _allS*; sleep 1; done;
WANT TO RETRY
################

Need to retry just run below two commands (make sure you dont have any other _all files that are important or any important greps going on – if you do delete and kill process manually)

Kills all the greps and removes all of the _all serach result files

killall -9 grep
rm -rf _all*
PUT ALL RESULTS TOGETHER
#########################

Run this after, it will put all of the _allSS files together

DST1=”_allTOGETHER_.txt”; for i in `ls | grep _allSS`; do echo “CONCATTING FILE: $i >> $DST1”; echo -e “TERM: $i\n###################” >> $DST1; cat $i >> $DST1; echo >> $DST1; done;

THE END
########

cat through all of the _all files

===============================================

Like templates of commands, maybe you use clcl (http://www.infotinks.com/windows—clcl—best-clipboard-manager) like I do. All of these commands are above.

GREP SEARCH TERMS TEMPLATE
##########################

CONTENTS: build terms, kill and remove last trys, multithread run, multithread run solaris, watch processes, while loop processes, then same processes commands with line numbers instead, put all together

cat << EOF > terms
term1
term2
term3
EOF

killall -9 grep
rm -rf _all*

RELDIR1=”searchthruthisdir”; for i in `cat terms`; do echo “=======================”; date; echo “Looking for: $i”; echo “=======================”; (time (grep -nir $i $RELDIR1) >& _allSS_$i.txt &); echo “———————-“;echo “Started PID: $!”;(ps awfuxx | egrep grep;); echo; done; echo “##############################”; echo “FINAL JOBS:”; (ps awfuxx | egrep grep;)

RELDIR1=”searchthruthisdir”;
for i in `cat terms`;
do echo “=======================”;
date;
echo “Looking for: $i”;
echo “=======================”;
(grep -ir $i $RELDIR1 >& _allSS_$i.txt &);
echo “———————-“;
echo “Started PID: $!”;
ps ax | egrep grep;
echo;
done;
echo “##############################”;
echo “FINAL JOBS:”;
ps ax | egrep grep;

watch ‘echo “PROCESSES:”; echo “===========”; (ps awfuxx | egrep grep;); echo; echo “RESULTS:”; echo “===========”; tail _all*;’

while true; do clear; “=====`date`======”; echo “PROCESSES:”; echo “===========”; (ps awfuxx | egrep grep;); echo; echo “RESULTS:”; echo “===========”; tail _all*; sleep 1; done;

watch ‘echo “PROCESSES:”; echo “===========”; (ps awfuxx | egrep grep;); echo; echo “RESULTS (number of lines):”; echo “==========================”; wc -l _allS*;’

while true; do clear; “=====`date`======”; echo “PROCESSES:”; echo “===========”; (ps awfuxx | egrep grep;); echo; echo “RESULTS (number of lines):”; echo “==========================”; wc -l _allS*; sleep 1; done;

DST1=”_allTOGETHER_.txt”; for i in `ls | grep _allSS`; do echo “CONCATTING FILE: $i >> $DST1”; echo -e “TERM: $i\n###################” >> $DST1; cat $i >> $DST1; echo >> $DST1; done;

Leave a Reply

Your email address will not be published. Required fields are marked *