Xfce 4.6 has some nice new features

Xfce, a popular lightweight desktop environment, has some nice new features in version 4.6 writes Arstechnica.

I’ve used Xfce many times — usually with VNC as the lightweight environment works much better over VNC than other “heavy” environments.

COTD: tee - read from standard input and write to standard output and file

Today’s COTD is the “tee” command.

Have you ever wanted to redirect the output of a script or program to a file, but be able to view it right away? Without putting the file in the background, and then tailing the file?

The tee command allows you to do just that.

Here is how we could use it:

$ nmap 192.168.123.0/24 | tee nmap.out

Starting Nmap 4.62 ( http://nmap.org ) at 2009-03-03 21:51 CST
Interesting ports on firewall.domain.com (192.168.123.1):
Not shown: 1687 closed ports
PORT STATE SERVICE
1/tcp open tcpmux
11/tcp open systat
15/tcp open netstat
22/tcp open ssh
23/tcp open telnet
79/tcp open finger
.
.
.
Nmap done: 256 IP addresses (14 hosts up) scanned in 147.576 seconds

$ ls -l
total 6231
-rw-r–r– 1 root root 5756 2009-03-03 21:55 nmap.out

Here we tell the shell to run the nmap command, send the output to the tee command, which then displays it and writes it to a file.  Tee also supports the “-a” option which will tell it to append to the filename specified if it already exists, otherwise, it will overwrite the file each time.

It’s important to note that tee only works with “stdin” and “stdout”.  It does not process “stderr”.  Stderr will still go to your terminal, but will not be captured in the file that tee is writing to.

COTD: touch - change file timestamps

Todays COTD (Command Of The Day) is the “touch” command.

The touch command is used to update the access and modification times of files.  However, I see it more commonly used to create a new file.  There are many times when you want to create a new file, but not actually put anything in it.  For example, if you have a script that checks for the presence of a file to control some logic.  In Solaris, if you “touch” a file called “/reconfigure”, it will perform a reconfiguration boot the next time you reboot the system.

Touch can also be used to update the access, or modification times, of a file.  This might be useful if you are using a file’s access or modification time to see if other files are newer than this reference file.

For example:

$ touch file1
$ touch file2
$ ls -l
total 0
-rw-r–r– 1 tuz tuz 0 2009-03-02 05:07 file1
-rw-r–r– 1 tuz tuz 0 2009-03-02 05:07 file2

Here we have created two files called file1 and file2.

$ touch timestamp
$ ls -l
total 0
-rw-r–r– 1 tuz tuz 0 2009-03-02 05:07 file1
-rw-r–r– 1 tuz tuz 0 2009-03-02 05:07 file2
-rw-r–r– 1 tuz tuz 0 2009-03-02 05:09 timestamp

We now have three files. Now, lets run a find command and look for files that are newer than our timestamp file.

$ find . -cnewer timestamp

There are none found. This is no surprise. Now, lets create another new file, and then run the same find command again.

$ touch file3
$ find . -cnewer timestamp
.
./file3

This time, find shows us the new file that is newer than the reference file. This could be handy for a number of uses.  For example, you could use this to quickly find out what files have changed since the last time you created a backup of your home directory, among many other uses.

Find also has the ability to search files files that have been accessed more recently than a reference file.    Please check the man page out for your local version of touch.

Some other options for touch include:

  • Change only the access time
  • Set the date/time to a user defined date/time
  • Change only modification time

Please check the man page for your local version of touch as some versions may be different.

COTD: netstat - Print network connections, routing tables, interface statistics, masquerade connections, and multicast memberships

The COTD for today is the netstat command.

The netstat command prints network connections, routing tables, interface statistics,  and a bunch of other goodies that are helpful in managing any UNIX box.

One of the common uses of netstat is to show the systems routing table.  This is done by the “netstat -r” command.

$ netstat -nr
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
192.168.123.0   0.0.0.0         255.255.255.0   U         0 0          0 eth0
0.0.0.0         192.168.123.1   0.0.0.0         UG        0 0          0 eth0

I add the -n on to the command as well.  This tells it to not try to reverse-lookup the hostnames that go with each address.  When I’m looking at a routing table, it makes more sense to see numbers anyways.

The output of this command shows that, in a general sense,  there are two routes configured.  One, a route for the network that this host is connected to, which says that all traffic for this network can just go out on the ethernet interface that it is connected to, and a default route.  The default route is where this host should send traffic for any destination that is NOT on the network that this host is connected to.   This is generally traffic going our to the Internet, but could also be another one of your networks which is behind a router.

Another use of the netstat command is the “netstat -a” command.  The netstat -a command shows all listening, and non-listening sockets.  This will show you TCP/IP ports that are listening on your server for incoming connections, and will also show connections that are already established.

For example:

$ netstat -an
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address               Foreign Address             State
tcp        0      0 0.0.0.0:8080                0.0.0.0:*                   LISTEN
tcp        0      0 0.0.0.0:21                  0.0.0.0:*                   LISTEN
tcp        0      0 0.0.0.0:22                  0.0.0.0:*                   LISTEN
tcp        0      0 0.0.0.0:23                  0.0.0.0:*                   LISTEN
tcp        0      0 0.0.0.0:25                  0.0.0.0:*                   LISTEN
tcp        0      0 192.168.123.30:22           192.168.123.56:1558         ESTABLISHED
tcp        0   2304 192.168.123.30:22           192.168.123.51:58528        ESTABLISHED
.

.

.

Ative UNIX domain sockets (servers and established)
Proto RefCnt Flags       Type       State         I-Node Path
unix  2      [ ACC ]     STREAM     LISTENING     5568   /tmp/.font-unix/fs7100
unix  2      [ ACC ]     STREAM     LISTENING     43084661 /tmp/mysql.sock
unix  2      [ ACC ]     STREAM     LISTENING     31759372 /var/run/cups/cups.sock

In the first part of this output, we see that we have the following TCP/IP ports listening; 8080, 21, 22, 23, and 25.  Also, there are two connections established from two other hosts on the network, to port 22.  Port 22, as you know, is SSH.   If you noticed that I also included the -n option here as well.  If I you omit the -n, it will reverse-lookup the DNS hostname for the IP address as well as show you what the port is defined as being used for in the /etc/services file.  For example:

$ netstat -a
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address               Foreign Address             State
tcp        0      0 *:webcache                  *:*                         LISTEN
tcp        0      0 *:ftp                       *:*                         LISTEN
tcp        0      0 *:ssh                       *:*                         LISTEN
tcp        0      0 *:telnet                    *:*                         LISTEN
tcp        0      0 *:smtp                      *:*                         LISTEN
tcp        0      0 host.domain.com:ssh    host.domain.com:1558    ESTABLISHED
tcp        0    832 host.domain.com:ssh    host.domain.com:58528   ESTABLISHED

Keep in mind, that in the case of the ports, what name that shows up here, may not be important.  Like hostnames, they reference to the /etc/services file is done for  <!– @page { size: 8.5in 11in; margin: 0.79in } P { margin-bottom: 0.08in } –>convenience.  This is why I generally always include the -n option with netstat.

The bottom part of this output shows Active UNIX domain sockets which we will cover in more detail later.

One other feature of netstat I use, is the “netstat -i” option.  netstat -i shows you the interface statistics for all of your networking ports such as Ethernet, PPP, etc.

For example, take a look at the output from one of my firewalls:

# netstat -i
Kernel Interface table
Iface   MTU Met   RX-OK RX-ERR RX-DRP RX-OVR   TX-OK TX-ERR TX-DRP TX-OVR Flg
eth0   1500   045951474      0      0      083559997      0      0      0 BMRU
eth1   1500   081641759      0      0      048132087      0      0      0 BMRU
eth2   1500   021402459      0      0      016603968      0      0      0 BMRU
eth2:  1500   0     - no statistics available -                        BMRU
eth2:  1500   0     - no statistics available -                        BMRU
eth2:  1500   0     - no statistics available -                        BMRU
eth3   1500   0       0      0      0      0   64497      0      0      0 BMU
lo    16436   0       0      0      0      0       0      0      0      0 LRU
ppp0   1450   0       9      0      0      0      24      0      0      0 MOPRU

This output shows, for each interface, statistics about the packets sent, and received.  This is useful for locating collisions, and other errors that could lead to troubleshooting many things — like duplex/speed settings, etc.

Netstat has many options.  I’d recommend reading the man page for your particular version of netstat.  Questions?  Feel free to ask via the comments page.

COTD - Command Of The Day

Welcome to COTD.  Each day I am going to pick a UNIX command and spend a little time sharing a little bit about that command.  I am going to call this “COTD” for “Command Of The Day”.

I may not get into every possible use of the selected command, but I am hoping to raise some awareness to some of the commands that are often forgotten about, or known about in the first place.

If you have a UNIX/Linux command you’d like to see featured here, please let me know by leaving a comment on this page.

Thanks!

Getting rid of all your serial consoles

If you work with UNIX servers that have serial consoles, and have terminals attached to each one, free up some space with a console server.

I’ve worked at companies with several hundred UNIX servers with each having their own serial attached terminal.   However, there is a better solution which many companies are utilizing.

Enter the console server.  Whether you have one server, or several thousand, a console server can make your life better.

There are two main uses for a serial console server.

  1. Eliminate the need to have one serial terminal connected to each system
  2. Provide the ability to access the console over the network (from your desk, from home at 2am, or from a beach in Cancun).

How does this work?  A serial console server is a hardware device that, with the very basic configuration, has one or more serial ports as well as a network connection.   The UNIX servers are connected to the serial ports, the network connection to your network, and then from any computer on the network, you can then access the console of any of your UNIX servers.  Typically this is with Telnet, SSH, or via a Web Browser.

Optionally, you can also add a few serial terminals to the console server so you can just hop on a terminal, select the port (server) you want to connect to, so you don’t have to run back to your desk.  You can also attach modems so you can dial-in (if everything else fails) to your console server to gain access to your consoles.

There are many more things you can do with console servers, and their usage is not limited ot just UNIX servers.  Network equipment such as routers, firewalls, load-balancers, or anything with a serial console port, can also be used.

Personally, I have an older Cyclads TS-1000 (16 port serial console server) that I purchased on eBay several years ago.  I use it in my lab to connect to the consoles of my servers and network gear.

Some companies that I would recommend, based on my past use, include:

Since these are serial console servers, they are best suited for UNIX hardware that have serial consoles.   For PC-Based UNIX/Linux servers, that have a standard Video/Keyboard/Mouse console, a KVM is best suited for these.  However, a few servers allow the console to be redirected to serial, which could allow a serial console server to be used for them as well.

$99 Linux Wall-Wart Computer!

O.K., this is now on my wishlist!  I absolutely love seeing Linux embedded in just about everything.  I just ran across this $99 Development Kit for a Linux based computer in a package about the size of a Wall-Wart.

It’s called the SheevaPlug and comes with a 1.2GHz processing, 512MB of DDR2 memory, 512MB of NAND based FLASH memory, USB, and Gigabit Ethernet.

The only thing that would be really cool, would be built-in 802.11 WIFI.

One-Third of Dell Netbooks running Linux

Laptop Magazine reports that one third of Dell’s Inspiron Mini 9s that have been sold are running Linux.  I’m still up in the air about what netbook I’m going to go with — but it will definitely be running Linux/UNIX.

I’d really like to see a Dell Latitude line of netbooks.   Inspiron, to me, just screams “consumer grade”.

Automatically Backing Up Your Google Docs

I love Google.  Google has so many wonderful products and services that really add value to my life, and business.  One of these products, is Google Docs.

Google Docs is really neat, but anyone keeping all of their data in “the cloud” should be concerned.  Anything could happen to the cloud… and with a little rain, all of our data could be washed away.

I’ve tried a few solutions to manually backup my Google documents.  But remembering to backup your files regularly, is not an option.

So after some digging around, I found some tools that will allow you to setup regular Google Docs backups of your files.

What you’ll need:

Python is probably already installed on your Linux distribution.  If not, you will need to install it.  The Google Data API is available in several programming langauges.  I chose Python for reasons unknown.  The gdatacopier is a python script that talks to the Google Data API to talk to Google.

I decided that I would just install this for my user account on my server, and not for the whole server.  We will first download the Google Data API and gdatacopier bundles from their respective websites, and extract them. I’ve not captured the output of the following commands.  I encourage you to review the readmes with each package before starting.

Download two archives with wget:

$ wget http://gdatacopier.googlecode.com/files/gdatacopier-1.0.2.tar.gz

$ wget http://gdata-python-client.googlecode.com/files/gdata.py-1.2.4.tar.gz

Uncompress them:

$ gzip -d gdatacopier-1.0.2.tar.gz

$ gzip -d gdata.py-1.2.4.tar.gz

Un-tar them:

$ tar xf gdatacopier-1.0.2.tar

$ tar xf gdata.py-1.2.4.tar

Install the Google Data API:

$ cd gdata.py-1.2.4

$ python setup.py install –home=~

$ cd ..

Copy the gdatacopier library to $HOME/lib/python/ and create a symbolic link.

$ cp -r gdatacopier-1.0.2 ~/lib/python/

$ cd ~/lib/python

$ ln -s gdatacopier-1.0.2 gdatacopier

Next, you’ll want to add a line in your .profile or .bash_profile.  You’ll want to edit an existing PYTHONPATH line, or create a new one:

PYTHONPATH=~/lib/python

export PYTHONPATH

Source in the profile by doing this:

$ . ./.profile

or

$ . ./.bash_profile

Finally, create a directory to store your Google Docs.

$ mkdir ~/GoogleDocsBackups

At this point, you should have the Google Data API and the gdatacopier library installed and ready to go.  I created a simple script to actually do the backups.  My script looks like this, and is located in a directory called “bin” created off of my home directory.

$ cat ~/bin/backup_googledocs
#!/bin/bash

# Source in the profile for good
# measure, since we'll be
#running this from cron.

. $HOME/.bash_profile

~/lib/python/gdatacopier/gdoc-cp.py \
–username ‘YourUserNameHere@gmail.com’ \
–password ‘YourPasswordHere’ \
–export default \
–google-id all \
–local ~/GoogleDocsBackups/

The script above basically exports ALL of your Google Docs Spreadsheets and Documents, and exports them as the default export format, which is Open Office format.   You can read the documentation for the libraries on their websites and learn about other things they can do.  The Google Data API, for example, lets you access all kinds of Google content.

Once you verify that your script works, and backs up your files successfully, you’ll want to schedule this in cron.  Otherwise, we still have a manual process that we’ll probably forget to do.

$ crontab -e

This will open your crontab in your editor.  You will want to add a line like the bottom line here:

SHELL=/bin/bash

# Backup my Google Docs each Day
10 6 * * * ~/bin/backup_googledocs

This will execute your script each day at 6:10am.  I hope this provides some comfort for you.  It does for me.  I no longer have to worry about having a backup copy of my Google Docs.

Bash Shell Version 4.0 released

The latest version of the Bash shell was recently released.  Version 4.0.   Bash is pretty much the default shell is most Linux distributions, and seems to be preferred by most Sys-Amins on other flavors of UNIX as well.

I just downloaded and compiled the source on my Ubuntu 8.10 workstation as I didn’t see it in the update stream yet.

The manual is available online as well.