====== Quick Command Set Daily Life ======

==== grab configure script from php installatio ====

<code bash>
php -i | grep configure | tr -d \'
</code>
==== find the disk usage for all the listed files/foldere ====

<code bash>
du -sckh * | sort -n
</code>
==== find uniq ip from file ====

<code bash>
cat /etc/sitemap |  awk '{print $2}' | xargs -n 1 ping -c 1 -t 1 | grep PING | awk '{print $3}' | sort | uniq
</code>

==== do something based on file list ====

<code bash>
while read k2;do <command> $k2;done < file
</code>

==== add text to end of each line ====

<code bash>
sed -i "s/$/<text to add>/" file
</code>
==== add text to start of each line ====

<code bash>
sed -i "s/^/<text to add>/" file
</code>
==== set terminal to color ====

<code bash>
TERM=xterm-color; export TERM
</code>
==== Break huge line based on character number ====
(specifically used for SSL)
<code bash>
sed 's/\(.\{64\}\)/\1\n/g' file
</code>
==== Search file and replace string ====

<code bash>
find ./ -name ".htaccess"|xargs perl -pi -e 's/original/newstring/g'
--OR--
find . -type f -print0|xargs -0 perl -pi -e 's/mysql_connect \(\"pqr.k2patel.com/mysql_connect \(\"xyz.k2patel.com/g'
--OR--
for file in *; do [[ $file = remove ]] && continue; fgrep -vf remove "$file" > k2.tmp ; mv k2.tmp "$file"; done
</code>

==== Load LVM Disk ====

<code bash>
vgscan --mknodes  //load the nodes in /etc
vgchange -ay    //activate the nodes
lvscan  //scan and check the status
mkdir /mnt/somename     //directory to mount
mount /dev/VolGroup00/LogVol00 /mnt/somename    //mount on directory
</code>

==== general variable goes to .htaccess ====

<code apache>
php_admin_flag safe_mode Off            //put this in the httpd.conf if nothing works

php_flag safe_mode Off
php_value safe_mode "0"
php_flag register_globals off

php_flag display_errors off

DirectoryIndex index.php index.shtml
ErrorDocument 404 /
ErrorDocument 403 /
ErrorDocument 500 /
ErrorDocument 302 /

ErrorDocument 403 http://www.k2patel.com

Redirect 301 / http://k2patel.com/show.php?m=1895&st=1&sid=65

php_flag log_errors on
php_flag safe_mode Off
php_flag register_globals on

php_value session.save_path /srv/www/apache/tmp
</code>

==== SSL redirect for https ====

<code apache>
RewriteEngine On
RewriteCond %{HTTPS} off
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}
</code>

==== Redirect all except one ====

<code apache>
RewriteEngine on
RewriteCond $1 !^the/pathto/the/directory
RewriteRule (.*) http://www.newsite.com/$1 [R=301,L]

OR

RewriteEngine on
RewriteRule !^the/pathto/the/directory - [C]
RewriteRule (.*) http://www.newsite.com/$1 [R=301,L]
</code>

==== lock file for no edit ====

<code bash>
chflags noschg <file name>  -- unlock
chflags schg <file name> -- lock
</code>

==== remove ^M from file ====

<code sh>
VI : %s/^M//g
(To get the ^M hold the control key, press V then M (Both while holding the control key) and the ^M will appear.)
-OR-
sed 's/'"$(printf '\015')"'$//
-OR-
tr -d '\r' <old.file >new.file
</code>

==== change file permission if it is not own by specific user ====

<code bash>
find . ! -user tiffany | xargs chown tiffany:tiffany
</code>

==== remove folder name `5 "` with find ====

<code bash>
find . -type d -name '5 "' -exec rm -Rv "{}" \;
</code>

==== add / remove IP alias ====

<code bash>
ifconfig fxp0 inet <IP> netmask <sub-net> alias        -- add
ifconfig fxp0 inet <IP> netmask <sub-net> -alias       -- remove
</code>

==== manipulate one file based on source and pattern ====

<code bash>
while read -r k2 ; do sed -e 's!\$k2!'"$k2"'!g' < site.cnf  ; done < sitelist.cnf > site.all.cnf
</code>

====  Line by Line two file merge ====

<code bash>
# -d used as delimiter in example it is space " ".
paste -d " " file1 file2 >"outputfile1"
</code>

==== find and command combination ====

<code bash>
for i in `find . -type f` ; do mv $i /home/yahoo/Photos/ ;done
</code>

==== how to download recursively folder / file using FTP ====

<code bash>
wget -r --no-clobber --continue --user=USERNAME --password=PASSWORD ftp://hostname
--OR--
ncftget -R ftp://ftp.mydomain.com  source dest
</code>

==== one line loop in linux ====

<code bash>
for i in `jot 251 4`; do echo xx.yy.pp.$i; done
--OR--
for ((i=4; i<=254; i++)); do echo "xx.yy.pp.$i"; done
--OR--
for i in {4..254}; do echo "xx.yy.pp.$i"; done
</code>

==== Move file if it contain following text ====

<code bash>
find . -type f -exec grep -q '*POSSIBLE SPAM*' {} \; -exec echo mv {} /home \;
--OR--
grep -rl 'POSSIBLE SPAM' |while read f; do mv "${f##*/}" "${DEST}"; done
--OR--
find . -type f -exec grep -Fq '*POSSIBLE SPAM*' -exec mv {} /home/ +  #maybe, or \;
</code>



==== How to set date on linux/unix System ====

<code bash>
date yymmddhhss
</code>

==== netstat with name of process only work on linux ====

<code bash>
netstat -tlpn
</code>

==== Get IP of the domains in the list ====

<code bash>
sed -e 's#.*#nslookup & |awk '"'"'/^Name/{n=$2}/^Address/{a=$2}($2=="canonical"){c=sprintf("%s%s",c,$1)}END{print "& "n" "c" "a}'"'"'#' hostlist.txt|sh
</code>

==== How to connect to serial connection ====

<code bash>
cu -l /dev/cuaa0
ttyd0 "/usr/libexec/getty std.9600"   vt100   on secure
ttyd0 "/usr/libexec/getty std.9600" dialup off secure
</code>

==== How to find Scripts on pages ====

<code bash>
find /srv/www/html -name '*.php' -o -name '*.htm*' -print0| xargs -0 grep -i '<script>function' | cut -d ':' -f1
</code>

==== How to mount cd image ====

<code bash>
mount -o loop disk1.iso /mnt/disk
</code>


==== How to modify files based on sample config and list of variables ====

it read sample config from sample.conf replace $k2 in that file with varlist and append all to file $k2.testing.cfg file

<code bash>
while read k2 ; do sed -e 's!\$k2!'"$k2"'!g' < /home/testing/sample.conf >> $k2.testing.cfg ; done < /home/testing/varlist
</code>


==== Test HTTPS Connection Response of $host ====
You may want to test a response of a HTTPS host using telnet port 443, however because of SSL, the httpd will not provide the information via telnet because of the lack of handshake for SSL certificate. You can use openssl as below:

<code bash>
openssl s_client -connect $host:443 -state -debug
GET / HTTP/1.0
Host: $host
</code>

==== Print Each word on next line ====

This will print each space separated word on new line.
<code bash>
tr ' ' '\n' < file
</code>


==== IP List For Asian Country based on range ====

In order to get all IP allocated to specific country you need to get list of IP Blocks.\\
you can get that list from following URL.
<code bash | IPv4>
wget ftp://ftp.apnic.net/public/apnic/dbase/data/country-ipv4.lst
</code>

<code bash | IPv6>
wget ftp://ftp.apnic.net/public/apnic/dbase/data/country-ipv6.lst
</code>

Now for IPv4, I never worked on IPv6 format so far so i am explaining IPv4 Only.\\
To generate list for all block from china you can simply use command as below.

<code bash>
cat country-ipv4.lst | awk -F : '{print $2, $3}' | grep cn | awk '{print $1}'
</code>

now you can save that file and use it.

==== MD5 Hashed Password ====

Many times mysql password or MD5 on phpmyadmin does not work on resetting password.
<code bash>
admin = 21232f297a57a5a743894a0e4a801fc3
secret = 5ebe2294ecd0e0f08eab7690d2a6ee69
OU812 = 7441de5382cf4fecbaa9a8c538e76783
</code>

==== Creating MD5 HASH ====
You can use following command in order to create MD5 HASH.

<code bash>
echo -n "9qL[%T#3SM" | md5sum
</code>
==== Output Redirection, Process Substitution & Tee ====

Show standard out to window and write to file
<code>
while $(/bin/true); do echo YAYYAYA; sleep 3; done > >(tee logfile)</code>

==== Email Once Exit Code EQ 1 ====
<code>
while $(/bin/sleep 300); do pgrep -u jthomas -f java; if [ $? -eq '1' ]; then echo "Render Done" | mail -s "Render Done" beep-jthomas@something.com; fi done &</code>

==== Output Users not in Passwd File ====

<code>
while read -r line; do grep $line /etc/passwd > /dev/null; if [ $? == 1 ]; then echo $line; fi;  done < /var/tmp/jtlist </code>

==== RPM Output in Nice Format ====
<code>
rpm -qa --queryformat='%{N}-%{V}-%{R}.%{arch}\n'</code>

==== Fixing permission of file based on RPM Spec ====
<code>
for k2 in $(rpm -qa); do rpm --setugids ${k2}; done 
for k2 in $(rpm -qa); do rpm --setperms ${k2}; done
</code>
==== Process Substitution with Sort and Sdiff ====
<code>
sdiff <(sort list) <(sort masterList)</code>

==== remove comments and blank lines from output ====

<code bash>
grep -v -e '^#\|^\W*$'
</code>
OR
<code bash>
grep '^[[:blank:]]*[^[:blank:]#;]' $file
</code>

==== Calculating Date ====
<code bash>
date -d '1 month 15 days 2 hour 30 minute ago'
</code>

==== Grab IP from ifconfig and ignore local ====
<code bash>
/sbin/ifconfig | grep 'inet ' | cut -d: -f2 | awk '{print $1}' | grep -v -e ^'\(127.0.0.1\|192.168.\|10.\)'
</code>

==== List All installed fonts ====
<code bash>
fc-list
OR
e.g. /usr/bin/fc-list :style=medium
</code>

==== Nail ====
Advanced Mail command which allow you to specify From.\\
it is now installed on most OS including RHEL / CentOS.

<code bash | Usage >
nail -eiIUdEFntBDNHRV~ -T FILE -u USER -h hops -r address -s SUBJECT -a FILE -q FILE -f FILE \
-A ACCOUNT -b USERS -c USERS -S OPTION users
e.g. nail -r k2patel@mail.us -s "testing message" k2patel@hotmail.com
</code>

==== File Tree ====
Print list of file in PWD
<code bash>
alias fltree="ls -R | grep ":$" | sed -e 's/:$//' -e 's/[^-][^\/]*\//--/g' -e 's/^/ /' -e 's/-/|/'"
fltree
</code>

==== For loop replace text with loop value ====
<code bash>
for k2 in {71..95} ; do sed -e 's!\$k2!'"$k2"'!g' <test2 ; done
</code>

==== Password hash Algorithm in linux ====
<code bash | Checking existing algorithm>
authconfig --test | grep hashing
</code>
<code bash | Changing to stronger one>
authconfig --passalgo=sha512 --update
</code>

==== sed to search and replace search text and number following ====
<code bash>
sed -ie '/serial/s/[0-9][0-9]*/2014050900/' 
</code>

==== Open postscript file to view ====
<code bash>
cat file.ps | ghostview -landscape -
</code>

==== Print yum environment ====
<code bash>
python -c 'import yum, pprint; yb = yum.YumBase(); pprint.pprint(yb.conf.yumvar, width=1)'
</code>

==== Perl Local::lib ====
Simple easy way to install perl module locally.

Simply install local::lib

In order to install module locally use command as shown below.
<code bash>
perl -MCPAN -Mlocal::lib -e 'CPAN::install(threads)'
</code>

==== Clearing cache linux ====
Make sure you run **//sync//** command before each command here.\\
Alternatively you can use
<code bash>
sysctl -w vm.drop_caches=<number>
</code>

**Free Dentries, Inodes and pagecache from cache memory**
<code bash>
echo 3 > /proc/sys/vm/drop_caches
</code>

**Free Inodes and Dentries from cache memory**
<code bash>
echo 2 > /proc/sys/vm/drop_caches
</code>
*Free PageCache from memory**
<code bash>
echo 1 > /proc/sys/vm/drop_caches
</code>

Writing to /proc/sys/vm/drop_caches allows one to request the kernel immediately drop as much clean cached data as possible.\\
This will usually result in some memory becoming more obviously available; however, under normal circumstances this should not be necessary.\\
BE WARNED that using vm.drop_caches can cause a deadlock if the system is under heavy memory and I/O load!!!

==== Ping packet size ====
<code bash>
ping -v -M do -s 8064 192.168.1.1
</code>

==== Remove file with ASCII Name ====

Simplest thing to do is use built in functionality to search and remove in command.
Here is the file i've on my machine.
<code bash>
�^?
</code>

which translate to //''$'\250\177'// <br>
So check or list all the files with ascii in it.
<code bash>
ls [^[:ascii:]]*
</code>

now to remove
<code bash>
rm [^[:ascii:]]*
</code>

:!: Regex can be modified to list all matching file before removal.

==== Testing connection or port without any software ====
In linux proper way to test tcp or udp port connection without any software installed should be creating dev node.

<code bash>
/dev/<proto>/<host>/<port>
</code>

out put can be redirected to tty with help of cat.
<code bash>
cat < /dev/tcp/k2patel.in/3306
</code>

==== Setting id tag on mp3 ====
Setting mp3 tag on multiple file based on filename as song name.\\
Same artist.
<code bash>
while read k2; do id3tag -a'xyz guru' -A'SuperSpeed' -s"${k2%%.*}" -y'2021' "${k2}" ; done < <(ls *.mp3)
</code>

==== FFMpeg Conversion of 4K ====
I had issue of no audio while playing file over DLNA.\\
I've did the transcoding so it can play audio on my Phillips 4K TV.

<code bash>
ffmpeg -i <input_file>.mp4  \
    -map 0:0 -map 0:1 -map 0:2 -map 0:3 -map 0:5 -map 0:12 \
	-c:v copy \
	-c:a:0 truehd -strict -2 \
	-c:a:1 ac3 \
	-c:s copy \
	<output_file>.mkv
</code>
OR
<code bash>
ffmpeg -i old_file.mp4 \
    -map 0:0 -map 0:1 -map 0:3 -map 0:5 \
    -c:v copy \
    -c:a:0 ac3 \
    -c:s copy \
    new_file.mp4
</code>
OR
<code bash>
ffmpeg -i old_file.mkv \
    -map 0:0 -map 0:1 -map 0:2 \
    -c:v copy \
    -c:a:0 ac3 \
    -c:s copy \
    -disposition:s:0 forced \
    new_file.mkv
</code>

  - First thing list all mapping on your existing file using 'ffmpeg -i <file>
  - Select all streams using map you would like to copy to your new file.
  - each map should be listed using map so i selected only 6 out of 12.
  - After that specify wich codecs to use.
  - in -c:v i'm copying video as is.
  - in -c:a:0 i'm converting audio to truehd
  - in -c:a:1 i'm also adding second audio stream which is ac3.
  - in -c:s i'm copying subtitles.