Modem Emulation.. Fast and easy!

I have a weekness for anything that communicates via RS-232 Serial.  This is probably due to most of my youth days being in front of a computer connected via modem to the world.

Occasionally I like to access Telnet BBS to re-live my youth.  Telnet works fine, but once you want to start doing file transfers, etc., it quickly becomes limited.

Enter “modemu”.

ModemU adds Telnet capability to a comm program by redirecting Telnet I/O to a PTY.  Basically, the Communications Program doesn’t really have any clue that it’s not a serial port.

You can get ModemU from Good ‘ol Sunsite by clicking here.

ftp://sunsite.unc.edu/pub/linux/apps/serialcomm/dialout/modemu-0.0.1.tar.gz

It compiled just fine on my Ubuntu 9.04 workstation after installing flex.  Once compiled, you can  use it as a stand-alone client, which doesn’t serve much purpose, if you ask me.

However, this is the way I used it with minicom:

$ modemu -c “minicom -p %s”

This should work with other communications program, as long as you can pass in the port to use.  ModemU, after creating the virtual PTY, passes in the PTY # to the command specified via the %s variable substitution.

Tada!

Once in minicom, you can do the following to connect to your favorite telnet systems:

atd”hostname

CONNECT

Welcome to….

To disconnect, once you exit your system, you will receive a “NO CARRIER” message.  Then, you can exit with at%q, or exit with your communications program.

* Main * 0:00:03 [2] DOVE-Net [1] General: /O

NO CARRIER
at%q

There you have it.  Simple Telnet<->Modem Emulation in Linux.  This should also work on other UNIXes, but your milage my vary.

BFTP: Back in the day….

Here is a little blast from the past for me.

Looking through my ~/bin directory, which is a collection of some of the scripts I’ve written going back to the wee days of me playing with UNIX.

I found this script, named “findjosh”.

#!/bin/sh

joshsip=`lynx -dump http://www.cae.wisc.edu/~resch/ip.txt | head -3 | tail -1`
case $? in
0) echo “Josh’s IP Address is presently “$joshsip;;
1) echo “..There was an error…”;;
esac

I know, it’s really simple, but I must have used this script a couple of times a day.   A little explanation.  This script was written in the very early 90’s.   This time was way before Instant Messaging that we know today.  To chat with friends back then, we often utilized the UNIX “Talk” facility, and later, Windows programs that provided the same functionality.

Instead of having a directory of people, or the ability to just punch in their “handle”, to chat with a friend, you had to know their IP address.  This would be easy, but even back then, they often changed quite often.

My friend Josh, as well as other friends from their era, resorted to many methods to let people know where they were.   In the case of Josh, he published his current IP address into a text file called ip.txt.  So whether he was in his dorm room, or on one of the numerous HP or SUN workstations, this made it easier to find him rather than using the “finger” command to hundreds of UNIX workstations!

Another version of this script that I was playing with is below:

#!/bin/sh

lynx -dump http://www.cae.wisc.edu/~resch/ip.txt > ~/bin/findjosh.tmp
cat ~/bin/findjosh.tmp | head -3 | tail -1

I wasn’t surprised to find a findjosh.tmp file in my bin directory:

$ cat findjosh.tmp

Josh last logged into the following address-
144.92.181.221
F18x-xxx.net.wisc.edu (fill in the x’s yourself)
-
Last updated-
08-31-1996 21:33:29

Does anyone remember doing stuff like this to track down your friends?  What methods did you use?

Simple Load Balancing with PV-Links [HPUX]

Many sites still only utilize HP’s PV-Links/Alternate-Links product to provide a simple failover for SAN connected disks. This product doesn’t do load balancing to spread out load, and is simply a failover if one of your SAN paths goes away.

This works great for redundancy, but with a little planning, you can accomplish simple load-balancing too.

Let’s consider a HPUX system which has two paths to the SAN, and two 50GB LUNS present to the server. The controllers are c17 and c19 and the two disks are t0d0 and t0d1. With PV-Links, you will thus have the following disks detected by the operating system:

/dev/dsk/c17t0d0
/dev/dsk/c17t0d1
/dev/dsk/c19t0d0
/dev/dsk/c19t0d1

Both c17t0d0 and c19t0d0 are the same LUN on the SAN, and also c17t0d1 and c19t0d1. With me so far?

The trick is, when adding these to a new volume group, you want to stagger them so that every disk you add, alternates between using the two controllers.

Therefore, I add them like this:

vgcreate vgTESTVG /dev/dsk/c17t0d0 /dev/dsk/c19t0d0
vgextend vgTESTVG /dev/dsk/c19t0d1 /dev/dsk/c17t0d1

You will end up with a vgdisplay -v /dev/vgTESTVG that looks like:

.
.
.
— Physical volumes —
PV Name /dev/dsk/c17t0d0
PV Name /dev/dsk/c19t0d0 Alternate Link
PV Status available
Total PE 12797
Free PE 12797
Autoswitch On
Proactive Polling On

PV Name /dev/dsk/c19t0d1
PV Name /dev/dsk/c17t0d1 Alternate Link
PV Status available
Total PE 12797
Free PE 12797
Autoswitch On
Proactive Polling On

Therefore, all disk access to the first disk will go down the c17 controller, and all access to the second disk will go down the c19 controller. You would then continue the pattern to add additional disks.

Keep in mind, that this is very basic load-balancing,  and might not even qualify as true load-balancing.  There are much better products out there that do a much better job (Powerpath, Veritias Storage Foundation, etc.).

However, if you only need very basic load balancing, and want to save a few dimes, this might work for you.

Cloning a Volume Group [hpux]

Often I am asked to clone an entire Volume Group including all of the logical volumes contained.  This can be a tedius task when the Volume Group contains a hundred or so logical volumes.

This request is often used for testing software releases, deployments, etc.

I’ve use this script at least a dozen times per month, and haven’t really fine tuned it, or improved it at all.  I still create the volume group manually, and add the physical disks.

# Loop through each logical volume name (They all should start with l),
# get the size, and then create in the new VG.

for i in `ls -1 /dev/vgORIGINALVG/l*`
do
MB=`lvdisplay /dev/vgORIGINALVG/$i | grep “LV Size” | cut -c23-40`
lvcreate -n $i -L $MB /dev/vgNEWVG
done

Real basic, but saves quit a bit of time.  I am merely keeping everything default, and am not taking into consideration any special settings.  Again, these are generally used for testing, and then removed — at least in my environment. So it’s not really a true “clone”.

I’m sure there are countless otherways to accomplish this, and maybe even an easier way, but I whipped this up one day when I had a request to clone 3 or 4 VG’s.

This might work on other LVM systems with some tweaking.

Fixing missing devices (NO_HW) on HPUX

This last weekend, after some SAN changes, some of our HPUX systems experienced problems where either one of the hardware path’s to disk devices changed or the only path on some tape devices changed.  Of course, this wasn’t found out about until Monday morning…

A reboot would have fixed this — but I don’t like rebooting UNIX/Linux unless it’s absolutely necessary (and it rarely is).

The first thing I noticed is that doing an ioscan -fnC tape showed “NO_HW” on one of our IBM tape drives.   NO_HW indicates the devices has dissapeared, and was there before.

Next, I forced HPUX to rescan for new devices with the insf command:

# insf -e -C tape

The “-e” option tells insf to reinstall the special files for pseudo-drivers and existing devices.   The “-C tape” told insf to only match devices that belonged to the tape class.   Since I like to narrow down any impact, I prefer to work with one class at a time.

Once this command was completed, I now saw the tape drive that was “NO_HW” earlier, appear in the ioscan output.

However, the one that went missing was still there at the old patch.   Now, again, a reboot would fix this…. But do we want to do that?  NO. You don’t.

The rmsf command comes to the rescue here.  Let’s say the one missing (NO_HW) was at the following hardware path:  0/2/0/0.197.12.255.1.8.0, the following command would remove it.

# rmsf -H 0/2/0/0.197.12.255.1.8.0

No reboot necessary.  There also ways to accomplish the same tasks under other OSes, and I will cover those in future posts.

Been a little crazy lately…

About 9 months ago I started a new project… Er, maybe even could call it a hack… A good hack.   Tuesday, it was born.

I’d like to introduce the newest UNIX guy into my family…   Meet my new Son, Dexter…

http://www.flickr.com/photos/47643618@N00/sets/72157622188318531/

Born 9:28am on 9/8/9.  8 pounds, 14 ounces.  20 1/2  Inches.

A very sad day…

Today marks a very sad day.  Last night, we experienced several extended power outages.   I’m saddened to say that “Nattie”, my HP DL380 RedHat ES4 Server, finally lost power.

No, she is O.K. now, and is back up and running.  I am saddened because this server had been up for 826 days and had been a braggin’ point for me quite sometime.  Sure, I’ve seen servers that have been up longer –but never has a machine at one of my “at home” Data Centers /Labs been up this long.

She was connected with redundant power supplies, with each one going to a separate commercial UPS, and each of those going to separate breakers, with each of those going to separate sides of the incoming 220 service.   There wasn’t much else I could have done to keep Nattie’s power redundant — short of a second power company feed.  :)

Right now, I am at the whim of the batteries in the UPS which surprisingly kept everything (several other servers, tape library, 4 switches, VPN Concentrator, Secure Computing Firewall, Backup Server, etc.) on for about an hour.   I’m definitely thinking a whole-house generator is going to be on my shopping list in the future!

10:09:03 up 800 days, 9:51, 3 users, load average: 0.33, 0.76, 0.62

(Uptime from July 13th, 2009, when I celebrated 800 days)

Well, at least now I have a good opportunity to add some additional memory and make a few other hardware changes!

Virtual Google Search Appliances? Sweet.

A good friend of mine, Eric Lightbody works for the University of Wisconsin Green Bay as a Web Developer. He was playing with their Google Mini Search Appliance (GSA), and I was getting jealous. I had played with the earlier Google Search Appliances a few years earlier at Time Warner Cable, but now I wanted to play with one again.

I was thinking… I wonder if they have a Virtual Appliance for people to play with it before buying one. A Google search later, I found what I was looking for.

Enter the Google Search Appliance Virtual Edition.  This is distributed by Google as a VMware Virtual Machine.   Within about an hour, I had my Virtual GSA up and running and indexing all of my websites including some sites that had NTLM based protected content.   Pretty neat.  It would have been much quicker, but the decompress process (p7zip) took like 30 minutes on a Core 2 Quad Ubuntu workstation, leaving me with about a 40GB VMDK.

The Virtual Edition is meant for development use.  Legally, you are not allowed to use it for any production use.  Plus, it’s limited to 50,000 indexed files.   I would highly recommend a real Google Mini search appliance.  But if you already have one,  the Virtual Edition would be great for testing website search integration changes before implementing them on your real GSA.  And, if you are thinking about getting one, trying out the Virtual Edition will allow you to get your feet wet, and experiment with it’s features.

I would strongly recommend giving it plenty of Virtual Resources when you fire this up.  I starved mine, at first, from the resources it wanted, and it performed a little sluggishly (to be expected).

Also, after booting it up, make sure you wait a bit before trying to connect to it.  It seemed to take a good 5 to 10 minutes before I could point a browser at it to connect — again, this could be related to me not giving it much memory.  I gave it 1GB, and the requirements said 3GB was the minimum needed.

Enjoy.

StorageTek 5320 Command Line Cheat Sheet

The StorageTek 5320 NAS Appliances are pretty neat devices.  I find it necessary to utilize the command line interface (CLI) quite often. Unfortunately, the documentation for the 5320’s CLI is pretty limited — at least I couldn’t find anything in the manual talking about each command.

So, for your enjoyment, I’ve summarized each command below.  Having everything on one page will save you the trouble of going through each one doing “help command” to find out what each does! (Which is what I seem to have to do every time I hop on one of these and try to remember which command does what).

Command Description Usage
? Alias for the help command
approve manipulate approve file approve [reload | list | test | help]
aratewrite write contents of a file (async), report performance aratewrite FILENAME [+OFFSET] TOTALKB [BLOCKSIZE] [MB_PER_COMMIT]
arp show arp table arp [-d] [ -a | DEST | -i Interf ]
audit Audit Enable Command audit –help for details
auditconfig Audit Configuration Command auditconfig –help for details
cacls Display an access control list (ACL) cacls pathname
calog Compliance Audit Log Utility calog –help
cat show contents of a file cat filename
chattr change extended attributes chattr [-RV] <[-+iumq] | [[+r] [[[[[cc]yy]mm]dd]HH]MM][.ss]>
chgrp change file group chgrp [-R] group file…
chkpnt checkpoint management command chkpnt sub-commands [parameters] …
chkpntabort summarily abort checkpoints on a volume chkpntabort VOLUME
chkpntls list checkpoints chkpntls VOLUME …
chkpntmk make (create) a checkpoint chkpntmk VOLUME CHECKPOINT-NAME
chkpntmv move (rename) a checkpoint chkpntmv VOLUME OLD-CHECKPOINT-NAME NEW-CHECKPOINT-NAME
chkpntoff stop checkpoints on volume(s) chkpntoff VOLUME …
chkpnton show checkpoint status / start checkpoints on volume(s) chkpnton [VOLUME ...]
chkpntrm remove (delete) a checkpoint chkpntrm VOLUME CHECKPOINT-NAME/PATTERN
chmod change file mode chmod [-R] mode file…
chown change file owner chown [-R] owner[:group] file…
chsmb Remove UNIX attribute update restrictions chsmb pathname
cls clear the screen cls
cp copy files cp [-v] [-c] [-r] src dst
crashlocks simulate file lock manager crash crashlocks
datalock show or reset multi-user rpc locks datalock {show | reset }
date display or set date and time date [[[[[cc]yy]mm]dd]HH]MM]
dbg Enable or disable the debugger dbg [ -i [ enable | disable ] | -s ]
del delete files del FILE …
df show (disk) volumes and free space df
diag Save diagnostic file in /dvol/diagnostic diag
disk show disks / edit disk partition table disk -help for details
dtq Manage directory tree quotas dtq help (for usage)
du estimate file space usage du [-s] [-h] [PATH] …
echo echo a message echo WORD …
em Set and show speed & duplex mode for Intel NIC em [set,show] NICname
exit exit command interpreter (close connection) exit
find find files bigger than size (in MB), owned by uid or gid find PATH [-name NAME] [-size SIZE] [-uid UID] [-gid GID]
fsck check file volumes fsck VOLUME …
fsctl Filesystem control command fsctl command [sub-command] volume
fszap Zap Partition fszap disk partition
grouplook lookup group by name or GID grouplook GROUPNAME/UID …
gunzip uncompress file gunzip FILE
gzip compress file gzip FILE
halt halt system halt [-fm]
hbainfo Display HBA Information hbainfo [-a]
help show available commands help [COMMAND ...]
hostlook lookup hosts by name/address hostlook HOSTNAME/ADDR …
hostmatch match hosts by name/address hostmatch HOSTPATTERN HOSTNAME/ADDR …
ioapic Program the IO-APIC ioapic prog/show/list/mask/umask apicid intpin [level polarity vector]
iscsi iSCSI service control iscsi
isp2suspend Suspend file system I/O for maintainance isp2suspend
keymon monitor keys keymon STOPCHAR
ldapcfg Configure NISLDAP client ldapcfg [-i | -d | -e]
ll long list directory contents (ls -lz) ll [-Ra] [dir]
lm License Manager Command lm –help for details
ln create hard/symbolic links ln [-s] target link
load report loaded modules/load a module load [MODULE]
loadvars load variables from values file loadvars
locks file lock manager report locks
logtail show last few log messages logtail [NLINE|all]
ls list directory contents ls [-Ralz] [file]
lsattr display extended attributes lsattr [-Radv]
luninfo Display LUN Information luninfo
maccnvrt Convert resource.frk directories maccnvrt share-name
man show contents of manpage with pagination man [-ah] [section] name …
mbstat print mbuf stats mbstat [-sv]
menu use the menus menu
mkdir make directories mkdir [-m mode] directory …
more show contents of a file with pagination more filename
mount Mount volume(s) mount [-rwfuh] [-o options] [VOLUME [...]]
ndmp NDMP service control ndmp
netserv set network service options netserv {enable|disable|status} {service name} [options]
netstat show routing/interface tables/network addresses and ports netstat [-rin]
nfsmount Inspect, establish, or change an NFS volume nfsmount [LOCALNAME [SERVER:PATH] [OPTION=VAL ...]]
nfsstat nfsstat nfsstat NFS statistics
nfsunmount NFS unmount a remote directory nfsunmount LOCALNAME
nic Set and show speed & duplex mode for Network NIC nic [set,show] NICname
niscfg Configure NIS+ client niscfg [ -i | -k ]
oplocks Configure oplocks oplocks [enable | disable] volume-name
password set password for admin password
ping ping hosts by name/address ping [-c count] HOSTNAME/ADDR …
poolstat show object pool status poolstat poolname …
praudit Audit Print Command praudit –help for details
pwtest test admin password pwtest
quit alias for exit quit (see exit)
quota show a quota / set quota limits quota [-g|-u] /VOLUME|/* NAME|ID|NAME/ID [bh=KB] [bs=KB] [fh=N] [fs=N]
ratecopy copy a file, report performance ratecopy SOURCE_FILENAME DEST_FILENAME [BLOCKSIZE]
ratecpu cpu performance tests ratecpu
ratemem memory performance tests ratemem
ratenet rate network performance ratenet HOSTNAME/ADDR [NSEND [SIZE [MAXFAIL]]]
rateread read contents of a file, report performance rateread FILENAME [+OFFSET] [[BLOCKSIZE] [TOTALKB]]
ratewrite write contents of a file, report performance ratewrite FILENAME [+OFFSET] TOTALKB [BLOCKSIZE]
reboot reboot system reboot [-fm]
recover perform head/controller recovery recover {head|ctlr}
repquota report quotas repquota [-g][-u][-n][-e|-E][-q|-v] VOLUME|QUOTAFILE …
revoke revoke client mounts revoke HOSTNAME/ADDR …
rm remove files rm [-v] [-r] path
rmdir remove directories rmdir directory …
rotatelog rotates partner log file rotatelog
route manipulate routing table route [-n] COMMAND [ [ MODIFIERS ] [ DEST GATEWAY ] ]
rpcinfo report RPC information rpcinfo
savecore save a core dump of the operating system savecore savecore [ -chi | -fpkmv directory ]
savevars save variables to values file savevars
scanvars scan variables for printer, modem, and network changes scanvars
sdate display the secure date sdate [-d]
set set a system variable set NAME VALUE
show show system variables show [NAME/PATTERN ...]
shutdown shutdown the system shutdown [-fm]
snmp SNMP trap destination control snmp
source read commands from a file source FILENAME
sp show stack trace for thread ID sp Thread_ID
stat stat files stat FILE …
status system status report status [hi|si|ai|ti|dm|fd|fs|rm|up|mm|all ...]
strmstat report list of associated named streams strmstat file-path
su login in as privileged user su
sync_sys Synchoronize system files sync_sys Source-Head-ID
syslog set local syslog parameters syslog timestamp=[0|1|2] debug=[on|off]
tar TAR Command tar –help for details
telnet telnet to host by name/address telnet HOSTNAME/ADDR
top Display active threads top [-b] [-i secs] [-t cnt] [-l]
tsp Display threads and their stack tsp [-s] [-t cnt] [-l] thread_name
umount Unmount volume(s) umount VOLUME [...]
uname print system information uname
unload unload a module unload MODULE
unset unset a system variable unset NAME
uptime show how long system has been up and cpu usage uptime
userlook lookup user by name or UID userlook USERNAME/UID …
version show software version version
vscan Virus Scan command vscan {help|check|stats}
xj extended journal functions xj -help for details
zic timezone utility zic [help | zone | dumpzones | list | ]

The Command Line Interface (CLI) is always handy, and usually the first place I check out. It’s especially handy in a clustered environment as you can login to one of your NAS heads, and then telnet from there to the heartbeat interface of the other NAS head. This is useful when your volumes are still failed over to the surviving NAS head along with it’s main IP address.

The 5320’s “menu” command will bring up a nice menu that lets you do most (all?) of the things you can do in the GUI.

One other note about the GUI. It appears that only one person can be logged in at a time. So if you are in it, and keep getting disconnected, it’s probably because someone else is trying to access it too. What you are seeing is a ping-pong effect of two (or more) people getting kicked out, and logging back in. Utilizing the CLI over telnet or SSH does not have this problem.

I may go into details about some of these commands in the future.

Enjoy!

Determining if your kernel and hardware is 32bit or 64bit

I’m often asked by end-users, developers, and fellow system administrators, how to determine if the installed OS is 32bit or 64bit.   I’m also asked if the hardware is capable of 64 bit, 32 bit, or both.   Here is how to tel by OS:

HPUX

# getconf KERNEL_BITS
64

This will tell you if your currently running kernel is 64 bits or 32 bits.  It returns the number of bits used by the kernel for pointer and long data types.  Current values include 32 and 64.

# getconf HW_32_64_CAPABLE
1

Returns which kernel is supported on the hardware.

You can also test this with a short C program:

# cat check3264.c
#include <stdio.h>
#include <unistd.h>

main()
{
long ret = sysconf(_SC_HW_32_64_CAPABLE);

if (_SYSTEM_SUPPORTS_ILP32OS(ret) != 0) {
printf(”system supports 32-bit OS\n”);
}

if (_SYSTEM_SUPPORTS_LP64OS(ret) != 0) {
printf(”system supports 64-bit OS\n”);
}
}
# cc check3264.c
# ./a.out
system supports 64-bit OS
#

In this example, we see the test system, an HP L2000-36 running HPUX 11.11, only supports a 64 bit HPUX Kernel.

# getconf HW_CPU_SUPP_BITS
64

This will show you if the CPU’s are capable of running 32, 64, or 32/64 bit kernels.

Solaris

For solaris, the isainfo -v should, at least for versions past 2.6,  show you want you need:

# isainfo -v
32-bit i386 applications

This may also say “sparc” if you are running on the sparc architecture. If you were on a 64 bit system, you would see something like this:

64-bit sparcv9 applications
32-bit sparc applications

This would be telling you that you are running a 64 bit kernel and can run either 64-bit sparcv9 applications, or 32-bit sparc applications.

To determine the cpu’s bit size capabilities, isainfo -b does the trick here too:

# isainfo -b
64

AIX

For AIX, we will use the bootinfo command.

bootinfo -y
32

This show is the hardware is 32 bit or 64 capable.

bootinfo -K
32

Shows the running kernel’s bit size.

Linux

For linux, we will look at the cpuinfo from /proc. Here, we are mainly interested in the “flags” for the CPU’s:

# cat /proc/cpuinfo | grep -i flags
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good pni dtes64 monitor ds_cpl est tm2 ssse3 cx16 xtpr pdcm sse4_1 lahf_lm

We are interested in three values in the output, as they indicate the bit size capabilities of the CPU:

16 Bit = rm (Real Mode)
32 Bit = tm (Transparent Mode)
64 Bit = lm (Long Mode)

This doesn’t necessarily mean the Motherboard is capable of 64 bit.

To determine the bit size of your running kernel, you can also use getconf, similar to HPUX, to find this info:

# getconf LONG_BIT
64

This shows that my kernel is running 64 bit.