Steve Ackman wrote:

Right now it's a hodgepodge of a shell script using
awk and sed and temporary files to populate the
database. The process has a problem with special
characters, throwing errors on the INSERT with 46
of every 10,000 posts in rcm.
Most of those seem to be in X-Trace headers. To get
around that for the short term, I *could* simply
deactivate the insertion of that header for the
articles that fail, but that's an ugly hack, and it
makes the database that much less useful when the
X-Trace header is the one you're particularly
interested in.

I'll have to google up what that is this weekend. Xtrace.

I may end up starting over with php or some other
language that containerizes the data in a way that
insulates it from the shell.

At any rate, it's that time of year that I start
spending less time at the computer, so I don't expect
to be putting much time into it until winter rolls
around again.

I know that feeling. Maybe next year I'll learn perl for real. Right now I
am relying on what I know about other languages to keep Gary Knights script
running. Add to the fact that everything is running inside of windows.

Still waiting for summer though. Ran the furnace last night.

Wes