Debugging on part(s) of the script

Debugging on part(s) of the script

and yes it is wierd that it is backwards – is on + is off

& man bash

fuse rpms

encfs ~/.data ~/data

sshfs bob@jrdoffice:/home/bob/Ike /Gandalf/RemoteSites/Ike

sudo mount -t cifs '//Ariel/Easy' ~/Easy -o credentials=/Gandalf/configs/.what,uid=500,gid=500
subnet scans

sudo ping -b 10.0.1.0

sudo nmap -v 10.0.1.0/16

Ch 4: Working with Files
Everything in a Linux file system can be viewed as a file (data files, directories, devices, pipes, etc)
Regular files: (20% loc 764)

file somefilename --determine type of file

touch /home/bob/newfile.ext -- create blank file

> /home/bob/newfile.txt -- create blank file

ls -l /usr/bin/apropos

file /usr/bin/whatis

file /bin/ls
directories

mkdir

umask umask -S (23% loc 852)

ln -s /path/somefile.txt /newpath/symlink.txt

symbolic link – own set of permissions, can exist on different partitions, new inode number
ln /path/file.txt /newpath/hardlink.txt

hard link – same permissions, cannot exist on different partitions, same inode number

symlinks ./ -- show all symbolic links in current dir

symlinks -r ./

symlinks -rv ./

named pipes & sockets overview only (22% loc 807)

421421421 -- rwxrwxrwx -- usergroupother

original permssions new

chmod 0700 any rwx------

chmod 0711 any rwx—x--x

chmod go+r rwx------ rwxr—r--

chmod 0777 any rwxrwxrwx

chmod a=rwx any rwxrwxrwx

chmod a+rwx any rwxrwxrwx
chmod -R 700 recursive

first 0 in all above = set-UID = 4, set-GID = 2, sticky = 1 (

set-UID will now work for shell scripts
only on ext2, ext3, ext4 file systems (24% loc 900)

lsattr, chattr --- a (append only), c (compressed), d (no dump), i (immutable), j (data journaling), s (secure deletion), t (no merging), u (undeletable), A ( no atime updates), D (synchronous directory updates), S (synchronous updates), T (top of directory hierarchy)

good to check the attributes once in a while for security purposes

Ownership

chown bob test/

chown bob:bob

chown -R bob /

cd or cd ~ -- change to user home directory

cd - -- change to previous directory

cd /tmp -- change to tmp off of root

cd tmp -- change to tmp off of current dir

cd .. -- change to parent dir

cp -a /var/www/html /backupdisk

cp -R /var/www/html /backupdisk

backup methods

dd (24% loc 879)

as root:

/sbin/fdisk /dev/hda -l > $BACKUPDIR/hda_partition_table.txt

Searching for files (25 % loc 917)

updatedb

locate & locate -i & locate -r (regluar expression)

which

find / -name e100 (25% loc 925)

ls -l, ls -la, ls -t, ls -i etc (26% loc 955)

alias ll="ls -lh"

alias la="ls -lah"

alias cl="cd /var/lo"
md5sum someFile.txt (26% loc 964)

sha1sum someFile.txt

sha1sum -c SHA1SUM.txt

lsof ---list open files

filelight ---diskusage
tripwire
Ch 5: Manipulating Text
Regular Expressions

a* any set of characters. a, ab, abc, aefopq

. any single character. a.c matches abc adc aqc

[ ] Matches a single character in the brackets a[bcd]e abe ace ade

[^ ] Matches a single character not in the brackets a[^bc]e aqe ade

^a a at the beginning of a line

*a$ a at the end of a line

a.c three character string starting with a and ending with c

[bcf]at bat, cat, or fat

[a-d]at aat, bat, dat ...

[A-D]at Aat ...

1[3-5]7 137, 147, 157

\tHello a tab character preceding the word Hello

\.[tT][xX][Tt] txt, Txt, TXt ...

vi, vim (http://vimdoc.sourceforge.net), joe, emacs, pico, nano

cat myfile.txt

cat myfile.txt > newcopy.txt

cat myfile.txt >> append.txt

cat -s myfile.txt display consecutive blank lines as one

cat -n myfile.txt show numbers on lines

cat -b myfile.txt show numbers on non blank lines
head myfile

cat myfile | head

head -n 10 myfile

ps auxw | head -10

tail -n 25 myfile

tail -f /var/log/httpd/access_log watch web server log continuously
more myfile.txt

less myfile.txt

/ repeat search

rpm -qa | sort | pr - -column=2 | less

Searching for text

grep francois myfile.txt

grep 404 /var/log/httpd/access_log

ps auwx | grep init

ps auwx | grep “\[*\]”

grep -Rn xdg /etc - directory tree with line numbers in result
Sorting output

rpm -qa | grep kernel | sort

rpm -qa | grep kernel | sort -r reverse order

ps auxw | sort -k 4,4

ps auxw | sort -k 2,2n
Replacing text with sed

cat myfile.txt | sed s/christopher/chris/

sed s/christopher/chris/ < myfile.txt > newmyfile.txt
Checking for differences between files with diff

diff /etc/named.conf.rpmnew /etc/named.conf

diff -u f1.txt f2.txt -- adds modification dates and times to output
seq 1 15 > f1.txt

sed s/4/four/ < f1.txt > f2.txt

vimdiff f1.txt f2.txt -- opens files side by side in vim
Using awk to process columns

ps auxw | awk '{print $1 $11}' --only show columns 1 & 11

ps auxw | awk '/bob/ {print $1, $11}' --show bob's processes
Converting text files to different Formats

unix2dos < f1.txt > f2.txt

dos2unix < f2.txt > f1.txt

Other

swap, boot, root

ext3 == ext2 + journaling

linux supports ext4, ext3, ext2, iso9660, Jffs21, jfs, msdos, ntfs, squashfs, swap, ufs, vfat, xfs

others nfs, sshfs, encfs, cifs & others (FUSE)
Partitioning:

install: used to be called Disk Druid

fdisk or parted
fdisk

/sbin/fdisk -l -- shows all partitions

(After Fedora 7 all IDE, SCSI, & SATA use /dev/sd..)

(newer Fedoras use the UUID – see the /etc/fstab file & /dev/disk

/sbin/fdisk -l /dev/sda

/sbin/fdisk /dev/sda --work on a particular disc

m --gets command listing

n --new partition (assumes ext3 type unless told otherwise)

d --delete partition

w --write changed info to disc (BE CAREFUL!)

parted

newer more functionality

1. 
   sudo /sbin/parted -l /dev/sda
   

Disk /dev/sda: 1000GB

Sector size (logical/physical): 512B/512B

Partition Table: msdos

Number Start End Size Type File system Flags

1 32.3kB 215GB 215GB primary ext3 boot

2 215GB 429GB 215GB primary ext3
changes immediately written to disk!

man parted shows brief listing info parted much more complete

both following will usually destroy file systems!

resize 2 will resize linux partitions (#2)

use the ntfsresize command to resize ntfs partitions

ntfsinfo

Both tools above only change parition table they do not format the partition

mkfs -t ext3 /dev/sda1

mkfs -t ext3 -v -c /dev/sda1 -- more verbose output and check for bad blocks

mkfs -t ntfs /dev/sda2

-- always put -t filesystemtype first

Working with existing partitions

Backup / Restore

sudo /sbin/sfdisk -d /dev/sda

# partition table of /dev/sda

unit: sectors
/dev/sda1 : start= 63, size=419424957, Id=83, bootable

/dev/sda2 : start=419425020, size=419425020, Id=83

/dev/sda3 : start= 0, size= 0, Id= 0

/dev/sda4 : start= 0, size= 0, Id= 0

/sbin/sfdisk -d /dev/sda | /dev/sdb -- copy to new disk

sudo /sbin/e2label /dev/sda1 yields /

sudo /sbin/e2label /dev/sda2 yields /1
/sbin/e2label /dev/sda2 /newlable
Virtual File System

portable, liveCD, virtual OS

du -sh mydisk & df -h (see below for more on both)

1001M mydisk

/sbin/mkfs -t ext3 mydisk

lots of info output

mkdir test

sudo mount -o loop mydisk test

mount

Viewing & Changing file system attributes

sudo /sbin/tune2fs -l /dev/sda1 (or dumpe2fs)

lots of information

man tune2fs

-c set maximal count before fsck

-j turn ext2 fs into ext3 by adding journaling
swap partitions

mkswap /dev/sda3

virtual partition as swap

dd -if=/dev/zero of=/tmp/swapfile count=65536

chmod 600 /tmp/swapfile

mkswap /tmp/swapfile

swapon

swapoff

/etc/fstab

LABEL=/ / ext3 defaults 1 1

devpts /dev/pts devpts gid=5,mode=620 0 0

sysfs /sys sysfs defaults 0 0

proc /proc proc defaults 0 0

LABEL=SWAP-sdc1 swap swap defaults 0 0

/dev/sdf1 /Gandalf/WinXP ntfs defaults 0 0

mount -o options

mount

mount, mount -t ext3, mount | sort, mount -l (labels)

ro, rw, uid=xxx, gid=xxx, noexec,

--bind (new additional location), --move

mount -v -o loop -t iso9660 diskboot.img ~/diskimg

mount -v -o loop local.iso ~/imgdir

/sbin/losetup -a -- show loopback device status

umount -v /dev/sda1

umount -v /Gandalf/Belinda
device is busy

/usr/sbin/lsof | grep mountpoint

/sbin/badblocks -v /dev/sdc1 readonly test

/sbin/badblocks -vsn /dev/sdc1 non destructive read write test (slowest)

/sbin/badblocks -vsw /dev/sdc1 faster destructive read write test

/sbin/fsck -TV /dev/sda1 do not display fsck version and be verbose

/sbin/fsck -TVy /dev/sda1 yes to all 'do I fix' questions
File system use

df -h usage summary in human readable mode

df -hi inode use also

df -hl only display local file systems

df -hT show file system type also

du -h /home/bob disk use of my home directory

du -h /home must be root

du -sh / summarize results

du -sch /home /data /usr/local multiple dirs

du -sh --exclude='*.iso' /home/bob exclude iso files from results & summarize

[-]A --catenate --concatenate

[-]c --create

[-]d --diff --compare

[-]r --append

[-]t --list

[-]u --update

[-]x --extract –get

-j --compress using bzip2

-z --compress using gzip

-v --verbose output
tar c *.txt | gzip -c > myfiles.tar.gz -- make tar archive then gzip it

tar czvf myfiles.tar.gz *.txt -- same thing

gunzip myfiles.tar.gz ; tar xf myfiles.tar

tar xzvf myfiles.tar.gz
tar tvf myfiles.tar -- list files in archive

tar -tzvf myfiles.tgs -- list files in gzip compressed archive

tar -Af archive1.tar archive2.tar -- adds archive2 to archive1

tar –delete file1.txt myfiles.tar -- deletes file from archive

lzop, gzip, bzip2 -- in order from fastest / least compression

rar x -- extract

rar a -- add file

tar xjvf myfiles.tar.bz2

gzip -v myfile -- verbose output

gzip -tv myfile.gz -- tests integrity of file

gzip -lv myfile.gz -- get detailed information

gzip -rv mydir -- compress all files in directory

bzip2 myfile -- myfile into myfile.bz2

bzip2 -v myfile

bunzip2 myfile.bz2

bzip2 -d myfile.bz2

bzip2 -vd myfile.bz2

backing up over network with ssh

rsnapshot vie yum install rsnapshot (http://www.rsnapshot.org/)

mkdir mybackup ; cd mybackup -- all files beginning with myfile are

ssh bob@server1 'tar cf – myfile*' | tar xvf - -- copied from server into local home dir
tar cf – myfile* | ssh bob@server1 'cd /home/bob/myfolder ; tar xvf - -- OUT
ssh bob@server1 'tar czf – myfile*' | cat > myfiles.tgz -- IN

tar czvf – myfile* | ssh bob@server1 ' cat > myfiles.tgz -- OUT

rsync -a source/ destination/ – equal to cp -a source/. destination/

rsync -a -e ssh source/ username@remotemachine.com:/path/to/destination/

--the -e option specifies the remote shell to use

rsync -a a/ b – gives b/foo point is backslashes matter but only on the source

rsync –delete –backup –backup_dir=bk-`date +%A` -avz test-src/ test-dest/$(date +%F)

rsync -avz –delete bob@server1:/home/bob/pics bobspics

mkdir /var/backups

rsync –delete –backup –backup_dir=/var/backups/backup-`date +%A` \

-avz bob@server1:/home/bob/Personal/ /var/backups/current-backup/

rm -rf /var/backups/backup-old/

mv /var/backups/backup-current/ /var/backups/backup-old/

rsync –delete –link-dest=/var/backups/backup-old/ -avz bob@server1:/home/bob/Personal \

/var/backups/backup-current/

• 
  longer script can be found here: http://samba.anu.edu.au/rsync/examples.html
  

-- rsync assumes that machine being backed up in only one where data is being modified

-- when have 2 (ie desktop & laptop) unison is better
yum install unison

unison /home/bob ssh://bob@server1//home/bob

unison /home/bob /mnt/backups/bob-home
-- to force unison to run in command line mode (-ui text)

unison /home/bob ssh://bob@server1//home/bob -ui text

unison /home/bob ssh://bob@server1//home/bob -auto

unison -help

unison -doc all | less

Backing up to removable media

mkisofs -o home.iso /home -- all files in DOS 8.3 naming mode

mkisofs -o home2.iso -J -R /home --Add Joliet & Rock Ridge extensions

mkisofs -o home3.iso -J -R music/ pics/ docs/ -- multiple dirs or files

-- /var/pics becomes /home/bob/Pictures on cd image

mkisofs -o home.iso -J -R -graft-points Pictures/=/var/pics/ /home/bob
-- add more information to ISO

mkisofs -o home.iso -R -J -p www.bob.org -publisher “Bob Thomas” -V “WebBackup” \

-A “mkisofs” -volset “1 of 4 backups, September 22, 2008” /home/bob
volname home.iso -- display volume name

isoinfo -d -i home.iso -- display all header information

mount -o loop home.iso /home/bob/test -- mount image in test dir

umount /home/bob/test
Burning to CD/DVD

cdrecord –scanbus -- shows information on CD/DVD drive(s)

cdrecord -v home.iso

cdrecord -v -eject home.iso

-- multisession using growisofs

growisofs -z /dev/sr0 -R -J /home/bob --Master & burn to DVD

growisofs -z /dev/sr0 -R -J /home/belinda -- Add to burn

growisofs -M /dev/sr0=/dev/zero -- Close burn
growisofs -dvd-compat -z /dev/sr0=home.iso -- burn image to DVD
CH 9: Checking and Managing Running Processes
Viewing active processes with ps

ps --help -- brief list of options

ps -A or e -- list all processes

ps -x -- list processes without controlling ttys

ps -u bob -- for user bob

ps -auwwx -- every process unlimited width BSD style

ps -axjf --

ps -ef --forest --

pstree
custom output with the -o option page 151

top -- show processes

top -d 5 -- change update delay from 3 to 5 sec

top -u bob -- show for user bob

top -n 10 -- update 10 times then quit

top -b -- run in non-interactive mode, good for file directed output

pgrep init -- yeilds ... why 3?

1

3204

pgrep -l init -- long listing

1 init

3204 start_kdeinit

sudo /sbin/fuser -mauv /home/bob -- show all processes with anything in /home/bob open

-- m show processes with file in . Open, v verbose, a all processes, u what user owns

sudo /sbin/fuser -k /boot -- kill every process that has anything in /boot open

-- sets process priority, regular user 19 (way low) to -20 (way high)

-- merely a suggestion

nice -n 12 gimp -- launch gimp with low priority

renice +2 -u bob -- set bob's process to lower priority

Running processes in background or forground with fg, bg, & jobs

open terminal, type gimp -- run gimp in foreground, will die if you close the terminal

type gimp & -- run gimp in background, ditto

--in running foreground process will stop it and put it in background

jobs --will list running process in that terminal

bg 1 --will put job 1 in background

fg 1 --will put job 1 in foreground

--kills current fg process

--kills terminal session
jobs -l -- long listing of all fg & bg process for current terminal session
kill & killall

ps -aux

kill 28665 -- send SIGTERM to process with PID of 28665

kill -9 4985 -- send SIGKILL to process with PID of 4985 (careful, no shutdown)

killall spamd -- kill all spamd running
Running processes away from the current shell

nohup gimp & -- run gimp with no ability to interrupt

Scheduling processes to run

at now +1 min

at>updatedb

at>Ctrl+d

at teatime

at now +5 days

at 10/05/08
atq -- query for jobs in queue
crontab -e -- create a crontab for current user and open in vi or vim

/etc/crontab -- minute, hour, day, month, & day of week

01 * * * * root run-parts /etc/cron.hourly

02 4 * * * root run-parts /etc/cron.daily

22 4 * * 0 root run-parts /etc/cron.weekly

42 4 1 * * root run-parts /etc/cron.monthly

-- simply link or put the script you want to run in one of the directories above

Ch 10: Managing the System
Focus in on Monitoring Resources in use

files in /proc (sudo ls -lah /proc)

might have to install sysstat packagel
Memory Use:

free (-m in megabytes, -g in gigabytes, -s 5 continuously display every 5 seconds)

free -m

free -m

total used free shared buffers cached

Mem: 8008 4846 3161 0 141 3793

-/+ buffers/cache: 912 7095

Swap: 16002 0 16002

top -- Shift M

vmstat -- view memory use over time

vmstat 3 -- update every three seconds

man vmstat -- field discriptions, watch for io backlog if lots memory in use, wasted CPU time

procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------

r b swpd free buff cache si so bi bo in cs us sy id wa st

CPU Usage:

iostat -c 3 -- update every 3 seconds

Linux 2.6.25.14-69.fc8 (Gandalf) 10/01/2008
avg-cpu: %user %nice %system %iowait %steal %idle

1.94 1.23 1.04 0.88 0.00 94.91

iostat -c -t -- print with time stamp

man iostat -- for listing of fields displayed

--> dstat -t -c 3 -- colors for different types of data

-----time----- ----total-cpu-usage----

date/time |usr sys idl wai hiq siq

01-10 17:08:41| 3 1 95 1 0 0

01-10 17:08:44| 0 1 99 0 0 0

01-10 17:08:47| 2 1 97 0 0 0

01-10 17:08:50| 0 1 99 0 0 0

01-10 17:08:53| 0 1 99 0 0 0

01-10 17:08:56| 0 1 99 0 0 0

01-10 17:08:58| 0 1 99 0 0 0

cat /proc/cpuinfo -- lots of info about processor(s)

flags line show features cpu supports

Storage Devices

du & df

iostat -d

Linux 2.6.25.14-69.fc8 (Gandalf) 10/01/2008

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn

sda 7.76 214.40 182.76 6445638 5494280

/usr/sbin/lsof -- all open files (lots)

lsof -c bash -- files open by bash shells

lsof -d cwd -- all directories open as current working dir in bash

lsof /dev/sda1 -- anything open on that filesystem

lsof /Gandalf/data -- anything open in that directory structure (and subs)
Mastering Time

system-config-date -- date, ntpd, timezone, etc gui

cat /etc/sysconfig/clock

# The ZONE parameter is only evaluated by system-config-date.

# The time zone of the system is defined by the contents of /etc/localtime.

ZONE="America/Chicago"

UTC=false

ARC=false

/usr/share/zoneinfo/America/Chicago -- time zone info

cp or ln -s above to /etc/localtime

--> date

Wed Oct 1 17:50:55 CDT 2008

--> date '+%A %B %d %G'

Wednesday October 01 2008

--> date --date='8 months 3 days'

Thu Jun 4 17:51:50 CDT 2009

date 081215212008 -- set date to Aug 12, 2:21pm 2008
cal -- show calendar

October 2008

Su Mo Tu We Th Fr Sa

1 2 3 4

5 6 7 8 9 10 11

12 13 14 15 16 17 18

19 20 21 22 23 24 25

26 27 28 29 30 31

--> cal 2009

2009
January February March

Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa

1 2 3 1 2 3 4 5 6 7 1 2 3 4 5 6 7

4 5 6 7 8 9 10 8 9 10 11 12 13 14 8 9 10 11 12 13 14

11 12 13 14 15 16 17 15 16 17 18 19 20 21 15 16 17 18 19 20 21

18 19 20 21 22 23 24 22 23 24 25 26 27 28 22 23 24 25 26 27 28

25 26 27 28 29 30 31 29 30 31

/sbin/hwclock -r -- display current CMOS hardware clock setting

/sbin/hwclock –hstosys -- set system clock from hardware clock (root)

Using Network Time Protocol

yum install ntpd

service ntpd start

chkconfig ntpd on

/etc/sysconfig/ntpd -- main config file

SYNC_HWCLOCK=no -- set to yes to sync

-- problem is why would you want to run a time server ?
ntpd -qg -- q says quit after syncing, g says don't panic for way off

Managing the boot process

A detailed look at the fedora boot process

BIOS

MBR on “first” bootable partition

GRUB

/boot/grub/grub.conf -- other configs are symbolic links to this

kernel

kernel needs root file system to load modules (block devices, etc)

devices drivers are on root file system so how does kernel get them ?

a small initial ram disk (initrd)

init process

/etc/inittab -- runlevel, etc

/boot/grub/grub.conf -- lots of other kernel boot options (table 2-1)

default=1

timeout=5

splashimage=(hd1,0)/boot/grub/splash.xpm.gz

title Fedora (2.6.26.3-14.fc8)

root (hd1,0)

kernel /boot/vmlinuz-2.6.26.3-14.fc8 ro root=LABEL=/ rhgb init=/sbin/bootchartd

initrd /boot/initrd-2.6.26.3-14.fc8.img

grub-install /dev/sda -- reinstall grub

mkinitrd ... -- recreate initial ram disk
Startup & Run Levels

/sbin/runlevel -- display current and previous

init 5 or 3 etc -- change runlevel

init q -- process changes in inittab (mostly for gettys)

/sbin/chkconfig --list, smb on, --add , --level ....
/sbin/service smb -- show usage statement

service smb restart -- etc

/etc/rc.d/rc
systemd

see /etc/systemd and /lib/systemd files

man systemctl

http://www.freedesktop.org/wiki/Software/systemd/FrequentlyAskedQuestions

The Kernel

uname

dmesg

lsmod

modinfo pata_acpi

/sbin/modprobe -l | grep c-qcam

modprobe c-qcam

modprobe -r c-qcam
/etc/sysctl.conf -- Kernel sysctl configuration file for Red Hat Linux

/sbin/sysctl -a | less -- list all kernel parameters

sudo /sbin/dmidecode -- list info about all hardware

sudo /sbin/hdparm /dev/sda -- view and change information relating to hard drive

Directory: 3351 -> Notes
Notes -> Fedora Core 5 Security Notes
3351 -> Bridge to success 1824 N. Pearl Street*Jacksonville, Florida 32209*(904) 520-7640 …i am Moving Forward! Graduation Information 2014

Download 295.94 Kb.

Share with your friends: