Table of Contents

Quick Command Set Daily Life

grab configure script from php installatio

php -i | grep configure | tr -d \'

find the disk usage for all the listed files/foldere

du -sckh * | sort -n

find uniq ip from file

cat /etc/sitemap |  awk '{print $2}' | xargs -n 1 ping -c 1 -t 1 | grep PING | awk '{print $3}' | sort | uniq

do something based on file list

while read k2;do <command> $k2;done < file

add text to end of each line

sed -i "s/$/<text to add>/" file

add text to start of each line

sed -i "s/^/<text to add>/" file

set terminal to color

TERM=xterm-color; export TERM

Break huge line based on character number

(specifically used for SSL)

sed 's/\(.\{64\}\)/\1\n/g' file

Search file and replace string

find ./ -name ".htaccess"|xargs perl -pi -e 's/original/newstring/g'
--OR--
find . -type f -print0|xargs -0 perl -pi -e 's/mysql_connect \(\"pqr.k2patel.com/mysql_connect \(\"xyz.k2patel.com/g'
--OR--
for file in *; do [[ $file = remove ]] && continue; fgrep -vf remove "$file" > k2.tmp ; mv k2.tmp "$file"; done

Load LVM Disk

vgscan --mknodes  //load the nodes in /etc
vgchange -ay    //activate the nodes
lvscan  //scan and check the status
mkdir /mnt/somename     //directory to mount
mount /dev/VolGroup00/LogVol00 /mnt/somename    //mount on directory

general variable goes to .htaccess

php_admin_flag safe_mode Off            //put this in the httpd.conf if nothing works
 
php_flag safe_mode Off
php_value safe_mode "0"
php_flag register_globals off
 
php_flag display_errors off
 
DirectoryIndex index.php index.shtml
ErrorDocument 404 /
ErrorDocument 403 /
ErrorDocument 500 /
ErrorDocument 302 /
 
ErrorDocument 403 http://www.k2patel.com
 
Redirect 301 / http://k2patel.com/show.php?m=1895&st=1&sid=65
 
php_flag log_errors on
php_flag safe_mode Off
php_flag register_globals on
 
php_value session.save_path /srv/www/apache/tmp

SSL redirect for https

RewriteEngine On
RewriteCond %{HTTPS} off
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}

Redirect all except one

RewriteEngine on
RewriteCond $1 !^the/pathto/the/directory
RewriteRule (.*) http://www.newsite.com/$1 [R=301,L]
 
OR
 
RewriteEngine on
RewriteRule !^the/pathto/the/directory - [C]
RewriteRule (.*) http://www.newsite.com/$1 [R=301,L]

lock file for no edit

chflags noschg <file name>  -- unlock
chflags schg <file name> -- lock

remove ^M from file

VI : %s/^M//g
(To get the ^M hold the control key, press V then M (Both while holding the control key) and the ^M will appear.)
-OR-
sed 's/'"$(printf '\015')"'$//
-OR-
tr -d '\r' <old.file >new.file

change file permission if it is not own by specific user

find . ! -user tiffany | xargs chown tiffany:tiffany

remove folder name `5 "` with find

find . -type d -name '5 "' -exec rm -Rv "{}" \;

add / remove IP alias

ifconfig fxp0 inet <IP> netmask <sub-net> alias        -- add
ifconfig fxp0 inet <IP> netmask <sub-net> -alias       -- remove

manipulate one file based on source and pattern

while read -r k2 ; do sed -e 's!\$k2!'"$k2"'!g' < site.cnf  ; done < sitelist.cnf > site.all.cnf

Line by Line two file merge

# -d used as delimiter in example it is space " ".
paste -d " " file1 file2 >"outputfile1"

find and command combination

for i in `find . -type f` ; do mv $i /home/yahoo/Photos/ ;done

how to download recursively folder / file using FTP

wget -r --no-clobber --continue --user=USERNAME --password=PASSWORD ftp://hostname
--OR--
ncftget -R ftp://ftp.mydomain.com  source dest

one line loop in linux

for i in `jot 251 4`; do echo xx.yy.pp.$i; done
--OR--
for ((i=4; i<=254; i++)); do echo "xx.yy.pp.$i"; done
--OR--
for i in {4..254}; do echo "xx.yy.pp.$i"; done

Move file if it contain following text

find . -type f -exec grep -q '*POSSIBLE SPAM*' {} \; -exec echo mv {} /home \;
--OR--
grep -rl 'POSSIBLE SPAM' |while read f; do mv "${f##*/}" "${DEST}"; done
--OR--
find . -type f -exec grep -Fq '*POSSIBLE SPAM*' -exec mv {} /home/ +  #maybe, or \;

How to set date on linux/unix System

date yymmddhhss

netstat with name of process only work on linux

netstat -tlpn

Get IP of the domains in the list

sed -e 's#.*#nslookup & |awk '"'"'/^Name/{n=$2}/^Address/{a=$2}($2=="canonical"){c=sprintf("%s%s",c,$1)}END{print "& "n" "c" "a}'"'"'#' hostlist.txt|sh

How to connect to serial connection

cu -l /dev/cuaa0
ttyd0 "/usr/libexec/getty std.9600"   vt100   on secure
ttyd0 "/usr/libexec/getty std.9600" dialup off secure

How to find Scripts on pages

find /srv/www/html -name '*.php' -o -name '*.htm*' -print0| xargs -0 grep -i '<script>function' | cut -d ':' -f1

How to mount cd image

mount -o loop disk1.iso /mnt/disk

How to modify files based on sample config and list of variables

it read sample config from sample.conf replace $k2 in that file with varlist and append all to file $k2.testing.cfg file

while read k2 ; do sed -e 's!\$k2!'"$k2"'!g' < /home/testing/sample.conf >> $k2.testing.cfg ; done < /home/testing/varlist

Test HTTPS Connection Response of $host

You may want to test a response of a HTTPS host using telnet port 443, however because of SSL, the httpd will not provide the information via telnet because of the lack of handshake for SSL certificate. You can use openssl as below:

openssl s_client -connect $host:443 -state -debug
GET / HTTP/1.0
Host: $host

This will print each space separated word on new line.

tr ' ' '\n' < file

IP List For Asian Country based on range

In order to get all IP allocated to specific country you need to get list of IP Blocks.
you can get that list from following URL.

| IPv4
wget ftp://ftp.apnic.net/public/apnic/dbase/data/country-ipv4.lst
| IPv6
wget ftp://ftp.apnic.net/public/apnic/dbase/data/country-ipv6.lst

Now for IPv4, I never worked on IPv6 format so far so i am explaining IPv4 Only.
To generate list for all block from china you can simply use command as below.

cat country-ipv4.lst | awk -F : '{print $2, $3}' | grep cn | awk '{print $1}'

now you can save that file and use it.

MD5 Hashed Password

Many times mysql password or MD5 on phpmyadmin does not work on resetting password.

admin = 21232f297a57a5a743894a0e4a801fc3
secret = 5ebe2294ecd0e0f08eab7690d2a6ee69
OU812 = 7441de5382cf4fecbaa9a8c538e76783

Creating MD5 HASH

You can use following command in order to create MD5 HASH.

echo -n "9qL[%T#3SM" | md5sum

Output Redirection, Process Substitution & Tee

Show standard out to window and write to file

while $(/bin/true); do echo YAYYAYA; sleep 3; done > >(tee logfile)

Email Once Exit Code EQ 1

while $(/bin/sleep 300); do pgrep -u jthomas -f java; if [ $? -eq '1' ]; then echo "Render Done" | mail -s "Render Done" beep-jthomas@something.com; fi done &

Output Users not in Passwd File

while read -r line; do grep $line /etc/passwd > /dev/null; if [ $? == 1 ]; then echo $line; fi;  done < /var/tmp/jtlist 

RPM Output in Nice Format

rpm -qa --queryformat='%{N}-%{V}-%{R}.%{arch}\n'

Fixing permission of file based on RPM Spec

for k2 in $(rpm -qa); do rpm --setugids ${k2}; done 
for k2 in $(rpm -qa); do rpm --setperms ${k2}; done

Process Substitution with Sort and Sdiff

sdiff <(sort list) <(sort masterList)

remove comments and blank lines from output

grep -v -e '^#\|^\W*$'

OR

grep '^[[:blank:]]*[^[:blank:]#;]' $file

Calculating Date

date -d '1 month 15 days 2 hour 30 minute ago'

Grab IP from ifconfig and ignore local

/sbin/ifconfig | grep 'inet ' | cut -d: -f2 | awk '{print $1}' | grep -v -e ^'\(127.0.0.1\|192.168.\|10.\)'

List All installed fonts

fc-list
OR
e.g. /usr/bin/fc-list :style=medium

Nail

Advanced Mail command which allow you to specify From.
it is now installed on most OS including RHEL / CentOS.

| Usage
nail -eiIUdEFntBDNHRV~ -T FILE -u USER -h hops -r address -s SUBJECT -a FILE -q FILE -f FILE \
-A ACCOUNT -b USERS -c USERS -S OPTION users
e.g. nail -r k2patel@mail.us -s "testing message" k2patel@hotmail.com

File Tree

Print list of file in PWD

alias fltree="ls -R | grep ":$" | sed -e 's/:$//' -e 's/[^-][^\/]*\//--/g' -e 's/^/ /' -e 's/-/|/'"
fltree

For loop replace text with loop value

for k2 in {71..95} ; do sed -e 's!\$k2!'"$k2"'!g' <test2 ; done

Password hash Algorithm in linux

| Checking existing algorithm
authconfig --test | grep hashing
| Changing to stronger one
authconfig --passalgo=sha512 --update

sed to search and replace search text and number following

sed -ie '/serial/s/[0-9][0-9]*/2014050900/' 

Open postscript file to view

cat file.ps | ghostview -landscape -
python -c 'import yum, pprint; yb = yum.YumBase(); pprint.pprint(yb.conf.yumvar, width=1)'

Perl Local::lib

Simple easy way to install perl module locally.

Simply install local::lib

In order to install module locally use command as shown below.

perl -MCPAN -Mlocal::lib -e 'CPAN::install(threads)'

Clearing cache linux

Make sure you run sync command before each command here.
Alternatively you can use

sysctl -w vm.drop_caches=<number>

Free Dentries, Inodes and pagecache from cache memory

echo 3 > /proc/sys/vm/drop_caches

Free Inodes and Dentries from cache memory

echo 2 > /proc/sys/vm/drop_caches

*Free PageCache from memory**

echo 1 > /proc/sys/vm/drop_caches

Writing to /proc/sys/vm/drop_caches allows one to request the kernel immediately drop as much clean cached data as possible.
This will usually result in some memory becoming more obviously available; however, under normal circumstances this should not be necessary.
BE WARNED that using vm.drop_caches can cause a deadlock if the system is under heavy memory and I/O load!!!

Ping packet size

ping -v -M do -s 8064 192.168.1.1

Remove file with ASCII Name

Simplest thing to do is use built in functionality to search and remove in command. Here is the file i've on my machine.

�^?

which translate to ''$'\250\177' <br> So check or list all the files with ascii in it.

ls [^[:ascii:]]*

now to remove

rm [^[:ascii:]]*

:!: Regex can be modified to list all matching file before removal.

Testing connection or port without any software

In linux proper way to test tcp or udp port connection without any software installed should be creating dev node.

/dev/<proto>/<host>/<port>

out put can be redirected to tty with help of cat.

cat < /dev/tcp/k2patel.in/3306

Setting id tag on mp3

Setting mp3 tag on multiple file based on filename as song name.
Same artist.

while read k2; do id3tag -a'xyz guru' -A'SuperSpeed' -s"${k2%%.*}" -y'2021' "${k2}" ; done < <(ls *.mp3)

FFMpeg Conversion of 4K

I had issue of no audio while playing file over DLNA.
I've did the transcoding so it can play audio on my Phillips 4K TV.

ffmpeg -i <input_file>.mp4  \
    -map 0:0 -map 0:1 -map 0:2 -map 0:3 -map 0:5 -map 0:12 \
	-c:v copy \
	-c:a:0 truehd -strict -2 \
	-c:a:1 ac3 \
	-c:s copy \
	<output_file>.mkv

OR

ffmpeg -i old_file.mp4 \
    -map 0:0 -map 0:1 -map 0:3 -map 0:5 \
    -c:v copy \
    -c:a:0 ac3 \
    -c:s copy \
    new_file.mp4

OR

ffmpeg -i old_file.mkv \
    -map 0:0 -map 0:1 -map 0:2 \
    -c:v copy \
    -c:a:0 ac3 \
    -c:s copy \
    -disposition:s:0 forced \
    new_file.mkv
  1. First thing list all mapping on your existing file using 'ffmpeg -i <file>
  2. Select all streams using map you would like to copy to your new file.
  3. each map should be listed using map so i selected only 6 out of 12.
  4. After that specify wich codecs to use.
  5. in -c:v i'm copying video as is.
  6. in -c:a:0 i'm converting audio to truehd
  7. in -c:a:1 i'm also adding second audio stream which is ac3.
  8. in -c:s i'm copying subtitles.