On Wed, 06 Jul 2011 17:28:01 -0700, Jeff Liebermann wrote:

On Wed, 6 Jul 2011 22:23:35 +0000 (UTC), Meat Plow
wrote:

Redundancy is your friend.

Well, not exactly. Cover thy ass, backups, and disk images are best.
RAID, mirroring, and tape backup are the road to hell. Been there, done
them all, and learned some really expensive lessons.

Redundancy means cover thy ass. Any way you can. Had my fair share of the
Dell PERC2 RAID controller battery failures and complete loss of RAID
containers. Dead Fujitsu drives on the second day of service for a
Poweredge server. I could go one and one...

I learned that
working on live computer systems for commercial clients that ran 3
shifts of CNC.

Ugh. Don't remind me. I had a customer running some APT program on
Lisa Xenix. They had two Lisa machines so theoretically, I was
protected. However, the system as stable as a house of playing cards.
Since Lisa Xenix wasn't getting updated, I didn't have that nightmare.
Instead, I had hardware failures, corrupted filesystems, and just plain
bad luck. So, to cover myself, it was:
find . -depth -print | cpio -odB | compress | \ rcmd machine_name "dd
of=/dev/rStp0"
before attacking. That saved my posterior more times than I care to
admit.

Most I remember about Xenix it was an AT&T product I think. Mine is all
NT4 server, NT4 Hydra, NT4 Terminal Server with Exchange, Citrix
Metaframe,
Windows Server 2k, 2003, Novell 3xx, 4.xx. BSD and linux for routers
(before the hardware appliances) Cobalt Cube mail servers, managed
switches, fiber the whole shebang.

Take them down for half an hour is a big deal.

I tried every variation of being considerate and nothing worked. Even
showing up at 2AM to do maintenance at the local hospital was a problem.
So, I went the other direction. I decided that I would set the
maintenance schedule and to hell with anyone that dared to interfere.
After checking with management, I would send out email and post signs
indicating that at 5:00PM, the servers would be down for maintenance.
At 5:00PM exactly, I would invoke the sacred incantations:
sync; sync; haltsys
and wait for the screaming to start. Invariably, someone had files
open, or unsaved data. I didn't care. Trust me, it works far better
than trying to be considerate.

Most of my nets were hybrid Novell/Windows. I sent a message to the
Novell clients to inform upon an impending shutdown so people could
save their work. Hell I charged by the hour so it didn't matter to me how
long I waited for everyone to back out. I could tell who was still in and
what files were open from the Novell server. I also had to do backups onto
Travan with open files for the corp systems running CNC 24/7. Skipping
open files wasn't an option. 21 tapes rotated weekely, one kept off site.
I never had an issue with viruses or malware. New virus definitions were
automatically deployed from the server running CA.

Screwing something
up big time and not having a redundancy plan to return before the
upgrade will lose you the client. Happened to me once on an old Novell
3.10 server running a golf course point of sale system. Upgraded the OS
to 3.12 and it died in the process. Later figured out after a couple
hours that the dos partition was full and that's where the boot files
are.

Ummm.... I think you mean 3.11. There was no 3.10. Next time, check
your disksapce with Volinfo and clean up marked for deletion files with
the purge command. Going from Novell 3.11 to 3.12 was a not major
project but did require having quite a bit of empty disk space.
incidentally, I have a (non-paying) customer running 3.12 on a 4GB
drive.

My bad, 3.11. Yeah I didn't check first. My mistake. I had been thrust
into Novell because the Novell guy quit after only two weeks training me.
I was learning Terminal Server at the same time. We had one problem
Terminal server where this guy didn't put the server into the install mode
before installing Exchange server on it. What a f'ing nightmare it was
rectifying all the errors. Took Microsoft paid support. After that it
never worked right. Memory leaks galore. I eventaully wiped it, Poweredge
2300? RAID 5 box dual Pentium 2's at 266 mhz IIRC LOL! The Novell 3.12
server was the same hardware. They ran FaxPress on it. Them people at the
steel plant expected me to be their god. It was rough for sure and I'm
sure you know the story. Everyone had their own special need. I'm trying
to link Macola ODBC, Pervasive SQL workstation clients with MS ODBC with
third party modules, FaxPress functions for faxing and emailing out of
Exchange with one keystroke. Stuff like that. Nightmare.

Deleted some
orphaned files, re-ran the upgrade and all was fine. Didn't lose the
client but they spent a lot of time entering sales they wrote down on
paper back into the system. I couldn't charge my time ethically so I
lost 300 bucks. That taught me a good lesson.

Well, if you only did that once, it's probably not a major disaster. In
my case, I'm always finding new ways to burn my time. Todays mess is a
classic. Virus infected laptop. Customer wants me to save some of the
junk in the Documents dumpster. She said it was ok to reformat and
start over, so I didn't see any reason to remove the virus. I copy the
files to a USB flash drive, and shove the flash drive into my main
office machine with the intent of scanning for viruses. I didn't have
to scan as autorun.exe conveniently installed the virus on my machine.
It took me about two hours to get rid of it (mostly spent scanning).
Argh.

LOL...sorry I use Win 7 inside Oracle Virtual machine for such things.
Have an install of Malwarebytes and Sophos on it. I don't do anything in
Windows except to keep it handy for remembering how it works. I do have a
netbook with Win 7 on it but hell I hardly use it. Mandriva 2010.2 is my
main OS, I love going to pages where they try to upload and run malware
like that virus checker crap. I feel very safe using linux. My first
linux box was a Slackware 3.xx box used without x-windows for a cable
modem router in my home. Again before the Linksys appliances were on the
market. Had to find experimental drivers for the 3Com 305 10/100 network
interfaces so that tells you how long ago that was



--
Live Fast Die Young, Leave A Pretty Corpse