Dirty Microsoft

Dirty Microsoft, they are making hotmail website not compatible with firefox 3

Web spiders and some basic concepts in flash drive file system

web spider: programs that automatically fetch information from websites, a email spam program is a example (though bad one) about web spiders/crawlers. but there are some good ones, such as RSS, a HTTP web spider that feeds the user news.

Ubuntu tools that can act as web spider:

wget - web downloading command line tool that automatically fetch the file indicated
snarf - simple web resource fetcher, just like wget

Linux Flash Drive File system

unlike disk file system, flash drives are made of NAND memory blocks and have some different characteristics:
1. although faster in reading and writing, flash drives memory's rewritable times are smaller than conventional hard disk, writing optimization algorithm is needed to lengthen the life span of flash drives, diverging programs from constantly writing to a frequently used part of the flash drive.
2. flash drive's writing is different, instead of being able to write 1's and 0's, flash drive writing only consist of 1's. thus the writing of a block requires the flash driver head to first erase everything into 0, reset the entire block once, then write the 1's. it also means whenever just a single bit has changed in a block of flash drive, the entire block must be rewritten, further decrease the life span of flash drives
3. memory dissapation after frequent reading also happens to flash drive. so after a certain amount of time of reading for a block of flash drive, the block must be rewritten to charge the NAND transisters in order to keep the memory, further increasing the frequency of rewriting

linux uses JFFS (journalling flash file system) and YAFFS for flash drive. they both employ writing frequency prevention algorithms.

mental note...

I need to write a script that downloads all the software i need with just one click... and possibly nice if i compile them on my machine from source code and then install it for every program

Useful Commands for Linux Administration

(copied from IBM developer networks)

1. fuser

check who is accessing a mounted volumn. fuser -k kills the process of the user that is accessing the mounted volumn

2. eject

ejects cdrom

3. mount /media/cdrom

mounts the cd manually

4. reset

resets the current console without having to restart the shell

5. su -

become another user, granting access privilege of that user

6. screen -s

screen shares with another person for one computer, one user need to be connected using ssh. it only works if both user are the same. screen can also split screens etc, and you can get out of the screen by pressing ctrl-A D, and then come back to the screen using the same command (screen -s

7. iperf - the linux ethernet speed test program

can get it from http://dast.nlanr.net/Projects/Iperf2.0/iperf-2.0.2.tar.gz

to run iperf as server for other machine to detect ethernet speed, use

iperf -s -f M

to connect to a iperf server in order to test ethernet speed, use

iperf -c -P 4 -f M -w 256k -t 60

test to connect to server with bandwidth 256k and test for 60 seconds


8. bash scriping using for loops, while loops, seq, awk, sort, uniq

some example
1)
# P=1; for i in $(seq -w 200); do echo "192.168.99.$P n$i"; P=$(expr $P + 1);
done >>/etc/hosts

connect to every local machine from 192.168.99.1 to 192.168.99.200 with computer name n001 to n200 and append them into etc/hosts file

# for num in $(seq -w 200); do ssh n$num free -tm | grep Mem | awk '{print $2}';
done | sort | uniq

connect to every machine from 192.168.99.1 to 192.168.99.200 with machine names n001 to n200 via ssh, grab the free memory in the machine from free command and print the free memory (second column) using awk, then pipe to sort them and pipe to take all unique numbers out

9. view processor information

cat /proc/cpuinfo

10. check number of processors

cat /proc/cpuinfo | grep processor | wc -l

11. grab BIOS information

dmidecode | less

note that dmidecode is difficult to grep

12. check driver for ethernet

ethtool -i eth0

Additionals:

GRUB boot option: press E in GRUB boot interface triggers editing option for booting command, add 1 after the kernel option will cause booting to single user mode. this is useful for admins lost their root password. once logged in as single user, use passwd to change the root password

SSH tunneling: you can tunnel through firewall using ssh to give access of a computer to networks outside using a intermediate machine. it takes 4 steps:

1)machine inside firewall ssh intermediate by the command ssh -R :localhost:22

2) while sshed into the intermediate, keep the connection alive by console script:

while [ 1 ]; do date; sleep 300; done

3) another machine connects to the intermediate using

ssh

4) the machine then ssh into machine inside firewall using

ssh -p root@localhost

it assumes you have root privilege in machine inside firewall

VNC tunneling (virtual network computing). VNC tunneling give the remote user a interface instead of console. to set it up takes 5 steps

1) start vnc server in machine inside firewall

vncserver -geometry 1024*768 -depth 24 :99

vncserver often starts on port 5900, thus :99 will open vncserver on 5999

2) machine inside firewall allows vnc forwarding to intermediate machine

ssh -R 5999:localhost:5999

at this time, the intermediate machine can view the machine inside firewall by

vncviewer localhost:99

3) keep the ssh open using

while [ 1 ]; do date; sleep 300; done

4) on the other machine that need to access the machine inside firewall, use this to connect to the intermediate.

ssh -L 5999:localhost:5999

the -L indicate only to pull information from the host, not to supply information (or pull, while -R indiate to push)

5) view the machine inside firewall by

vncviewer localhost:99

on sidenote, Putty in Windows can set the vnc port using user interface instead of command line in linux

Viewing error messages from programs during ssh: ssh doesn't report program errors when it is running. to view the program errors, you need to cat /dev/vcsl (or vcs1??)

Some useful network linux/unix commands

1. ping
used for detecting connection speed and status

2. nmap -A ...
used for detecting ports and services of the IPs. a very good network scanning tool

3. netcat
used to connect any IP and send/receive information from it

4. snort
packet logging and network traffic analysis tool

5. tcpdump
display tcp packet received by this computer from the network it has been attached to

6.kismet
network detector, packet sniffer and intrusion detection system. work with wireless

7. wireshark
network monitor tool

8. traceroute
detects the routers the packets went through as it is being transfered and returned from destination

9. telnet
telnet connection tool to connect to another computer through TCP

10. nslookup
domain name look up tool

11. john the ripper
password cracking software for unix