About Us

Don Y[_3_]

On 2/8/2016 4:20 PM, wrote:
On Mon, 08 Feb 2016 13:30:38 -0700, Don Y
wrote:

On 2/8/2016 12:45 PM, wrote:
On Mon, 08 Feb 2016 03:16:28 -0500, wrote:

On Sun, 07 Feb 2016 23:46:16 -0700, Don Y
wrote:

Then, you don't have customers who can drag you into court...

Anyone in that position who is seeking backup advice on a home owner
repair BB is a moron who should be sued.

In enterprise systems we used to sell "availability services" that
including all sorts of backup schemes including active mirrored sites
hundreds or even thousands of miles away but that has little to do
with a home user not losing their baby pictures.
In a commercial setting things like RAID controllers will be
standardized, if for no other reason to simplify parts logistics. They
will also have a strict backup regimen with onsite and offsite copies
of just about everything.
The critical stuff would be mirrored on two separate controllers and
backed up at least once a day to another media.

At the office we backup to a NAS box - both images of VMs and backups
of databases and folders - which are then duplicated onto a backup
server box, and all important stuff gets backed up from the NAS to
removeable hard drives - one for each Monfay to Thursday, and 5
separate Friday drives, rotated for the corresponding Friday of the
month.

How do you know that the data copied to the NAS accurately reflects the
data *sourced* to the NAS? How do you know that the data copied from the
NAS to those individual drives reflects the contents of the NAS? How
do you know that the drives are intact while sitting on a shelf, someplace?

Well, you are never 100% sure. but 99.99999999 is getting pretty
close. The backup programs we use all have verification and error
checking (ever hear of VEAM??)

We never know for sure the drive is intact sitting on the shelf, but
we have days before and after if one fails - and the drive is checked
for integrity before and during the backup. If any of the backup
sessions fails there are 3 of us get e-mails to tell us - and
thenetwork backup specialist that set up the system gets to find out
why.

My first "backup" device was an F880 9track drive (transport):
http://www.electrovalueinc.com/images/f880.jpg
On a 10 inch reel, I could get about 50MB without "compressing"
the data in any way. As the disk I was only 60MB, I would create
different "system images" and store each on a different tape.
So, if I was designing a circuit, I would "load" the system
that contained the schematic editor and PCB layout tools; if
I was writing software, I'd load the system image that contained
my assemblers/linkers/compilers/debuggers. If I was designing a
mechanical enclosure, I'd load the system image that contained
AutoCAD, etc.

As these images never needed to change, I never had to rewrite
*those* tapes, just "backup" any "working files" that I was
creating (a subset of the disk) -- onto a different tape!

But, tape needs to be "retensioned" periodically to prevent
print-through. So, I would just reread each tape (not actually
RESTORING any files in the process) to perform the retensioning
*and* verify the integrity of the data stored on the tape.

Storing an entire gigabyte was a significant undertaking!

As this technology was only marginally supported (due to the
controller I had purchased) under DOS, I eventually had to write
a device driver to use it to access those tapes when I moved to
Net/Free-BSD.

By that time, I had "upgraded" to an M990:
http://mail.sunstarco.com/images/cipher1.gif
about twice the size -- but much higher recording density so
~200MB/tape reel... using the exact same tapes!

Same retensioning problem. Same verification solution.
Virtually the same amount of time per reel -- but 4 times
the data involved.

I eventually migrated most of my "archive" to a set of four
BA356 "shelfs":
http://i.ebayimg.com/images/g/JeEAAOSwpDdVZ1YM/s-l300.jpg
(though that one is shown standing upright instead of sideways
in an equipment rack; also, two drive modules are missing).
This was housed in a short ~30 inch rack with casters beneath
so I could roll it in and out of the closet when not in use.

This gave me 28 SCSI (50 pin, SE) drives that I could, in
theory, bring online at the same time. In practice, I
could only power up one or two shelfs due to power and
heat concerns. Initially, it was populated with 4G drives
(e.g., 100GB!) but later upgraded to 9G drives (250GB!)

These were easier to verify: no "tapes to load", etc.
And, I could script the process to check 7 drives in
sequence, then alert me to swap to the next "shelf".

But, I needed the space that the little rack took up.
So, I discarded the rack and 3 of the shelfs (saving
the power supply modules as "spares" and, obviously,
the disk modules!) and kept most of the modules:
http://www.security-lab.de/pics/sw-disk-blue.JPG
on the shelf in the closet with the *disk* shelf permanently
connected to one of my computers.

The modules made handling the disk drives easier and more
reliable than dealing with "bare" drives. But, the verify
effort now required more of my involvement.

About the time that I was upgrading the 9G drives to 18G drives,
I moved to a D1000 array:
http://ep.yimg.com/ay/yhst-34967395916512/sun-storedge-d1000-disk-array-7.gif
Now, I could keep 12 drives spinning and cut the effort
required to swap drives in and out during the "verify"
operation. Also, the drive sleds were much smaller
than the "modules" used in the DEC shelf so I could stack
the off-line units in much less space than the DEC modules
had required!

This approach has currently left me with a pair of Sun 711 multipacks:
http://unixhq.com/websgt/1227808913_711.gif
each supporting twelve 146GB drives (1.5T) -- in much less space than
any of the previous "incarnations".

But, the drives are expensive and much higher performance than
is needed for an "archive". And, require a SCSI HBA to talk
to them -- something that puts an extra limitation on the
machines that I can use to access them. E.g., none of my
laptops have SCSI support (well, I have a PCMCIA SCSI adapter
but that's hokey).

Additionally, it requires ALL of the drives to spin up in order
to access *any* of them!

Meanwhile, the M990 gave way to a DLT2000xt (or, maybe it was the 2500?):
http://ep.yimg.com/ay/westernbackupsolutions/quantum-dlt-2000xt-15-30gb-scsi-tape-drive-4.jpg
that takes up 1/10 the volume! And, the 15/30G tapes are smaller
(and cheaper) than a 3.5" disk drive!

Over time, that drive was replaced with (presently) a set of
six DLT7000's (35/70GB).

But, again, lots of manual intervention required to do the
periodic verification, head cleanings, etc.

[I flirted with a DLT4000 changer after the DLT2000xt but it
was too bulky, noisey and always had me worried that it would
mangle a tape without my knowing!]

I've also played with various "DDS" cartridges, 8mm and 4mm
autoloaders, etc. I have stuff backed up on a wide variety of
media (including MO cartridges).

The current solution has been to adopt the "I" aspect of the
RAID acronym: redundant array of INEXPENSIVE disks! And,
given that this is an archive and not something that gets
hammered on regularly, the current best approach is consumer
grade, large capacity, external USB disks!

As damn near every machine I own has a USB port, this means
such a disk can be physically accessed from any of them.

If I treat them as free-standing entities, "loosely coupled"
(instead of tightly coupled in a REAL array), then they can
operate independent of each other. I can move them around
at will (unlike one of the disk arrays described above)

If I choose a well-supported file system format, then the
software interface won't require any special effort to store
or retrieve files!

But, with many TB available, keeping track of what's where
becomes a challenge! Hence, a database to record everything's
location.

Having that, I can then locate anything I want without having to
power up each drive, poke around in the hope of finding what I
want, then powering down and moving on to the next candidate.

And, once you have a list of all the files, you can augment
that with a list of their checksums! Now, instead of
doing byte-wise compares (to their backup images), you can
just look at the "current signature" to be assured that
you have an intact file.

Arranging for each to be scanned whenever it is powered up
(remembering where you had stopped your previous scan when
it was powered down), means you can KNOW that the file
is intact AND accessible. No need to drag disks or tapes
off a shelf and explicitly perform this action -- it can
all be automated (so it gets DONE, even if you dislike doing
it).

As it makes no assumptions about the actual medium (it could
be read-only!), it can be extended to include other types of
volumes, as well (e.g., all of my optical media are cataloged
though seldom "verified" -- cuz they are "masked" media, not
RW consumer media)

[I have a table of clipart thumbnails that lets me view
the hundreds of thousands of clipart images that I have on
store-bought optical media -- without having to keep printed
catalogs on hand *or* thumb through hundreds of CD's!
Just as easy to store an image in a database table as it is
to store a filename!]

I've been doing this for decades. I'm well aware of the
consequences and costs of the various "backup" technologies
that have come (and gone) over that time period. I am
*hoping* that this approach represents my last rehash
of the Archive capability. Hopefully, any future changes
just involve faster or bigger disks and processors to
talk to them!

Do you have a procedure in place where you regularly reassure yourself
that your process *works*? And, other procedures in place to handle
the case where you discover that it *doesn't* (didn't) work?

:
As I said, it is verified and reported on-the run - and yes, we do
occaisionlly attempt to restore a file from both the NAS and a random
removeable drive. We have several weeks of backup on the NAS -

We have never had a file we could not retieve.

My uncle ran a small CO. They had a small jet engine in a brick bunker
"out back" that powered a backup generator. So, if the AC mains failed,
they could keep the PSTN running off the "Battery" while the generator
came online.

The "system" was regularly tested to ensure the jet would fire on
demand, the genset operated, etc.

Testing cannot ensure something will work next time it is required. It
can just tell you it ran last time it was tested.

It can however pretty accurately predict it won't start next time if
it fails the test.

That's one of the appeals of my current solution! The files get
verified REGULARLY -- every copy of them -- without me having to
load tapes or disk modules off the shelf. Of course, there's no
guarantee that the next time the disk is powered up that it will
actually *spin* up. But, that potential exists for disk and
tape cartridges stored offline as well! At least I don't have
to worry that a tape has been damaged because it was stored in
the wrong orientation. And, if a medium is *starting* to get
flakey, I'll see it sooner rather than later!

Until, one day, when it didn't. : Thankfully, the only casualty
was the billing computer, which went offline during the "self-induced
outage".

Ooops!

Thread Tools	Search this Thread
Show Printable Version	Search this Thread: Advanced Search
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Completely OT	bm[_2_]	UK diy	11	March 26th 13 03:02 PM
Completely OT	harry	UK diy	7	December 4th 12 08:36 PM
Sound effects from Qbasic? (Tucker)	Robert Baer[_3_]	Electronic Schematics	0	November 27th 10 03:33 AM
And now, for something completely different -	charlieb	Woodworking	19	May 20th 07 05:20 PM
And now - something completely different.	charlieb	Woodturning	9	May 16th 07 04:13 AM

Menu

About Us