Some bots as you say - limit to 1 or 2 or 3 levels deep only.

Martin
Martin H. Eastburn
@ home at Lions' Lair with our computer lionslair at consolidated dot net
TSRA, Endowed; NRA LOH & Patron Member, Golden Eagle, Patriot's Medal.
NRA Second Amendment Task Force Charter Founder
IHMSA and NRA Metallic Silhouette maker & member.
http://lufkinced.com/


Ignoramus4791 wrote:
On 2008-08-31, Gunner Asch wrote:
www.httrack.com/

Been using it for years. Works on MOST websites, though not all.

If you try downloading my website algebra.com, you will get into an
infinite recursion through millions of pages. That's why I prevent
most such bots from accessing my site. This would work only on very
simple sites.



----== Posted via Pronews.Com - Unlimited-Unrestricted-Secure Usenet News==----
http://www.pronews.com The #1 Newsgroup Service in the World! 100,000 Newsgroups
---= - Total Privacy via Encryption =---