| Domains | Hosting | Web Design | Web Management | SME | Internet | Search Engines | Content | PageRank | Link Campaign | SEO Copy Writing | Keywords |
   

WebPageOne Solutions
Web Robots Directory

ABCdatos BotLink
Agent: ABCdatos BotLink/1.0.2 (test links)
Host: 217.126.39.167
Email: botlink+AEA-abcdatos.com
Acme.Spider
Agent: Due to a deficiency in Java it's not currently possible to set the User-Agent.
Host: *
Email: jef@acme.com
Ahoy! The Homepage Finder
Agent: 'Ahoy! The Homepage Finder'
Host: cs.washington.edu
Email: marclang@cs.washington.edu
Alkaline
Agent: AlkalineBOT
Host: *
Email: dblock@vestris.com
Anthill
Agent: AnthillV1.1
Host: anywhere
Email: info@anthill.org
Walhello appie
Agent: appie/1.1
Host: 213.10.10.116, 213.10.10.117, 213.10.10.118
Email: aimo@walhello.com
Arachnophilia
Agent: Arachnophilia
Host: halsoft.com
Email: taluskie@utpapa.ph.utexas.edu
Arale
Agent: no
Host: *
Email: flaviotordini@tiscali.it
Araneo
Agent: Araneo/0.7 (araneo@esperantisto.net; http://esperantisto.net)
Host: *.esperantisto.net
Email: araneo@esperantisto.net
AraybOt
Agent: AraybOt/1.0 (+http://www.araykoo.com/araybot.html)
Host: *
Email: robot@araykoo.com
ArchitextSpider
Agent: ArchitextSpider
Host: *.atext.com
Email: spider@atext.com
Aretha
Agent:
Host:
Email: davew@well.com
ARIADNE
Agent: Due to a deficiency in Java it's not currently possible to set the User-Agent.
Host: dbs.informatik.uni-muenchen.de
Email: Gross@dbs.informatik.uni-muenchen.de
arks
Agent: arks/1.0
Host: dpsindia.com
Email: aniruddha.c@usa.net
ASpider (Associative Spider)
Agent: ASpider/0.09
Host: nova.pvv.unit.no
Email: fredj@pvv.ntnu.no
ATN Worldwide
Agent: ATN_Worldwide
Host: www.allthatnet.com
Email: info@allthatnet.com
Atomz.com Search Robot
Agent: Atomz/1.0
Host: www.atomz.com
Email: mike@atomz.com
AURESYS
Agent: AURESYS/1.0
Host: crrm.univ-mrs.fr, 192.134.99.192
Email: mannina@crrm.univ-mrs.fr
BackRub
Agent: BackRub/*.*
Host: *.stanford.edu
Email: page@leland.stanford.edu
unnamed
Agent: BaySpider
Host:
Email: marki@baytsp.com
BBot
Agent: bbot/0.100
Host: *.netcologne.de
Email: poluf1 at yahoo dot co dot uk
Big Brother
Agent: Big Brother
Host: *
Email: Francois.Pottier@inria.fr
Bjaaland
Agent: Bjaaland/0.5
Host: barry.bitmovers.net
Email: tbray@textuality.com
BlackWidow
Agent: BlackWidow
Host: 140.190.65.*
Email: khooghee@marys.smumn.edu
Die Blinde Kuh
Agent: Die Blinde Kuh
Host: minerva.sozialwiss.uni-hamburg.de
Email: maschinist@blinde-kuh.de
Bloodhound
Agent: None
Host: *
Email: genius@ukonline.co.uk
Borg-Bot
Agent: borg-bot/0.9
Host: 24.11.13.173
Email: botdev@skunkfarm.com
BoxSeaBot
Agent: BoxSeaBot/0.5 (http://boxsea.com/crawler)
Host:
Email: boxseasearch@yahoo.com
bright.net caching robot
Agent: Mozilla/3.01 (compatible;)
Host: 209.143.1.46
Email:
BSpider
Agent: BSpider/1.0 libwww-perl/0.40
Host: 210.159.73.34, 210.159.73.35
Email: okumura@rsl.crl.fujixerox.co.jp
CACTVS Chemistry Spider
Agent: CACTVS Chemistry Spider
Host: utamaro.organik.uni-erlangen.de
Email: wdi@eros.ccc.uni-erlangen.de
Calif
Agent: Calif/0.6 (kosarev@tnps.net; http://www.tnps.dp.ua)
Host: cobra.tnps.dp.ua
Email: kosarev@tnps.net
Cassandra
Agent:
Host: www.aha.ru
Email: billy168@aha.ru
Digimarc Marcspider/CGI
Agent: Digimarc CGIReader/1.0
Host: 206.102.3.*
Email: wmreader@digimarc.com
Checkbot
Agent: Checkbot/x.xx LWP/5.x
Host: *
Email: graaff@xs4all.nl
ChristCrawler.com
Agent: Mozilla/4.0 (compatible; ChristCrawler.com, ChristCrawler@ChristCENTRAL.com)
Host: 64.51.218.*, 64.51.219.*, 12.107.236.*, 12.107.237.*
Email: jeremy.deyoung@christcentral.com
churl
Agent:
Host:
Email: yunke@umich.edu
cIeNcIaFiCcIoN.nEt
Agent: cIeNcIaFiCcIoN.nEt Spider (http://www.cienciaficcion.net)
Host: epervier.cqhost.net
Email: root@cyberdark.net
CMC/0.01
Agent: CMC/0.01
Host: haruna.next.ne.jp, 203.183.218.4
Email: shinobu@po.next.ne.jp
Collective
Agent: LWP
Host: *
Email: genius@ukonline.co.uk
Combine System
Agent: combine/0.0
Host: *.ub2.lu.se
Email: tsao@munin.ub2.lu.se
ConfuzzledBot
Agent: Confuzzledbot/X.X (+http://www.confuzzled.lu/bot/)
Host: *.ion.lu
Email: bot@confuzzled.lu
CoolBot
Agent: CoolBot
Host: www.suchmaschine21.de
Email: info@suchmaschine21.de
Web Core / Roots
Agent: root/0.1
Host: shiva.di.uminho.pt, from www.di.uminho.pt
Email: wc@di.uminho.pt
XYLEME Robot
Agent: cosmos/0.3
Host:
Email: preda@xyleme.com
Internet Cruiser Robot
Agent: Internet Cruiser Robot/2.1
Host: *.krstarica.com
Email: robot@krstarica.com
Cusco
Agent: Cusco/3.2
Host: *.cusco.pt, *.viatecla.pt
Email: clerigo@viatecla.pt
CyberSpyder Link Test
Agent: CyberSpyder/2.1
Host: *
Email: amant@cyberspyder.com
CydralSpider
Agent: CydralSpider/X.X (Cydral Web Image Search; http://www.cydral.com/)
Host: *.cydral.com
Email: cydral@cydral.com
Desert Realm Spider
Agent: DesertRealm.com; 0.2; [J];
Host: *
Email: spider@desertrealm.com
DeWeb(c) Katalog/Index
Agent: Deweb/1.01
Host: deweb.orbit.de
Email: dewebmaster@orbit.de
DienstSpider
Agent: dienstspider/1.0
Host: sappho.csi.forth.gr
Email: asidirop@csi.forth.gr
Digger
Agent: Digger/1.0 JDK/1.3.0
Host:
Email: admin@bulldozersoftware.com
Digital Integrity Robot
Agent: DIIbot
Host: digital-integrity.com
Email: robot@digital-integrity.com
Direct Hit Grabber
Agent: grabber
Host: *.directhit.com
Email: DirectHitGrabber@directhit.com
DNAbot
Agent: DNAbot/1.0
Host: xx.dnainc.co.jp
Email: tomatell@xx.dnainc.co.jp
DownLoad Express
Agent:
Host: *
Email: dlxpress@mediaone.net
DragonBot
Agent: DragonBot/1.0 libwww/5.0
Host: *.paczone.com
Email: admin@paczone.com
DWCP (Dridus' Web Cataloging Project)
Agent: DWCP/2.0
Host: *.dridus.com
Email: rmm@dridus.com
e-collector
Agent: LWP::
Host: *
Email: smarty@thatrobotsite.com
EbiNess
Agent: EbiNess/0.01a
Host:
Email: mdavis@kieser.net
EIT Link Verifier Robot
Agent: EIT-Link-Verifier-Robot/0.2
Host: *
Email: mcguire@eit.COM
ELFINBOT
Agent: elfinbot
Host: *.letsfinditnow.com
Email: admin@letsfinditnow.com
Emacs-w3 Search Engine
Agent: Emacs-w3/v[0-9\.]+
Host: *
Email: wmperry@spry.com
ananzi
Agent: EMC Spider
Host: bilbo.internal.empirical.com
Email: hpayne@u-media.com
esculapio
Agent: esculapio/1.1
Host: 80.34.92.45
Email: imasd@cype.com
Esther
Agent: esther
Host: *.falconsoft.com
Email: tim@falconsoft.com
Evliya Celebi
Agent: Evliya Celebi v0.151 - http://ilker.ulak.net.tr
Host: 193.140.83.*
Email: ilker@ulak.net.tr
nzexplorer
Agent: explorersearch
Host: bitz.co.nz
Email: paul@bourke.gen.nz
FastCrawler
Agent: FastCrawler 3.0.X (crawler@1klik.dk) - http://www.1klik.dk
Host: 1klik.dk
Email: crawler@1klik.dk
Fluid Dynamics Search Engine robot
Agent: Mozilla/4.0 (compatible: FDSE robot)
Host: yes
Email: zoltanm@nickname.net
Felix IDE
Agent: FelixIDE/1.0
Host: *
Email: felix@pentone.com
Wild Ferret Web Hopper #1, #2, #3
Agent: Hazel's Ferret Web hopper,
Host:
Email: ghbos@postoffice.worldnet.att.net
FetchRover
Agent: ESIRover v1.0
Host: *
Email: ken@engsoftware.com
fido
Agent: fido/0.9 Harvest/1.4.pl2
Host: fido.planetsearch.com, *.planetsearch.com, 206.64.113.*
Email: fido@planetsearch.com
Hämähäkki
Agent: Hämähäkki/0.2
Host: *.www.fi
Email: Timo.Metsala@www.fi
KIT-Fireball
Agent: KIT-Fireball/2.0 libwww/5.0a
Host: *.fireball.de
Email: info@fireball.de
Fish search
Agent: Fish-Search-Robot
Host: www.win.tue.nl
Email: debra@win.tue.nl
Fouineur
Agent: Mozilla/2.0 (compatible fouineur v2.0; fouineur.9bit.qc.ca)
Host: *
Email: jvandal@9bit.qc.ca
Robot Francoroute
Agent: Robot du CRIM 1.0a
Host: zorro.crim.ca
Email: maparent@crim.ca
Freecrawl
Agent: Freecrawl
Host: *.freeside.net
Email: ekhall@freeside.net
FunnelWeb
Agent: FunnelWeb-1.0
Host: earth.planets.com.au
Email: eaglesd@pc.com.au
gammaSpider, FocusedCrawler
Agent: gammaSpider xxxxxxx ()/
Host: *
Email: support@gammasite.com
gazz
Agent: gazz/1.0
Host: *.nttrd.com, *.infobee.ne.jp
Email: gazz@nttrd.com
GCreep
Agent: gcreep/1.0
Host: mbx.instrumentpolen.se
Email: anders@instrumentpolen.se
GetBot
Agent: ???
Host:
Email: zav@macromedia.com
GetURL
Agent: GetURL.rexx v1.05
Host: *
Email: James@Snark.apana.org.au
Golem
Agent: Golem/1.1
Host: *.quibble.com
Email: geoff@quibble.com
Googlebot
Agent: Googlebot/2.X (+http://www.googlebot.com/bot.html)
Host: googlebot.com
Email: googlebot@google.com
Grapnel/0.01 Experiment
Agent:
Host: varies
Email: v93_kat@ce.kth.se
Griffon
Agent: griffon/1.0
Host: *.navi.ocn.ne.jp
Email: griffon@super.navi.ocn.ne.jp
Gromit
Agent: Gromit/1.0
Host: *.austlii.edu.au
Email: dan@austlii.edu.au
Northern Light Gulliver
Agent: Gulliver/1.1
Host: scooby.northernlight.com, taz.northernlight.com, gulliver.northernlight.com
Email: crawler@northernlight.com
Gulper Bot
Agent: Gulper Web Bot 0.2.4 (www.ecsl.cs.sunysb.edu/~maxim/cgi-bin/Link/GulperBot)
Host: yuntis*.ecsl.cs.sunysb.edu
Email: gulperbot@ecsl.cs.sunysb.edu
HamBot
Agent:
Host: *.hamrad.com
Email: john@futureone.com
Harvest
Agent: yes
Host: bruno.cs.colorado.edu
Email:
havIndex
Agent: havIndex/X.xx[bxx]
Host: *
Email: havIndex@hav.com
HI (HTML Index) Search
Agent: AITCSRobot/1.1
Host:
Email: a94385@cs.ait.ac.th
Hometown Spider Pro
Agent: Hometown Spider Pro
Host: 63.195.193.17
Email: admin@hometownsingles.com
Wired Digital
Agent: wired-digital-newsbot/1.5
Host: gossip.hotwired.com
Email: bowen@hotwired.com
ht://Dig
Agent: htdig/3.1.0b2
Host: *
Email: andrew@contigo.com
HTMLgobble
Agent: HTMLgobble v2.2
Host: tp70.rz.uni-karlsruhe.de
Email: ley@rz.uni-karlsruhe.de
Hyper-Decontextualizer
Agent: no
Host:
Email: cliff@tricon.net
iajaBot
Agent: iajaBot/0.1
Host: *.scs.carleton.ca
Email: morin@scs.carleton.ca
IBM_Planetwide
Agent: IBM_Planetwide,
Host: www.ibm.com www2.ibm.com
Email: epc@www.ibm.com"
Popular Iconoclast
Agent: gestaltIconoclast/1.0 libwww-FM/2.17
Host: gestalt.sewanee.edu
Email: chris@gestalt.sewanee.edu
Ingrid
Agent: INGRID/0.1
Host: bart.ilse.nl
Email: ilse@ilse.nl
Imagelock
Agent: Mozilla 3.01 PBWF (Win95)
Host: 209.111.133.*
Email: belanger@imagelock.com
IncyWincy
Agent: IncyWincy/1.0b1
Host: osiris.sunderland.ac.uk
Email: simon.stobart@sunderland.ac.uk
Informant
Agent: Informant
Host: informant.dartmouth.edu
Email: info_adm@cosmo.dartmouth.edu
InfoSeek Robot 1.0
Agent: InfoSeek Robot 1.0
Host: corp-gw.infoseek.com
Email: stk@infoseek.com
Infoseek Sidewinder
Agent: Infoseek Sidewinder
Host:
Email: mna@infoseek.com
InfoSpiders
Agent: InfoSpiders/0.1
Host: *.ucsd.edu
Email: fil@cs.ucsd.edu
Inspector Web
Agent: inspectorwww/1.0 http://www.greenpac.com/inspectorwww.html
Host: www.corpsite.com, www.greenpac.com, 38.234.171.*
Email: doug@greenpac.com
IntelliAgent
Agent: 'IAGENT/1.0'
Host: sand.it.bond.edu.au
Email: s1523@sand.it.bond.edu.au
I, Robot
Agent: I Robot 0.4 (irobot@chaos.dk)
Host: *.mame.dk, 206.161.121.*
Email: irobot@chaos.dk
Iron33
Agent: Iron33/0.0
Host: *.folon.ueda.info.waseda.ac.jp, 133.9.215.*
Email: watanabe@ueda.info.waseda.ac.jp
Israeli-search
Agent: IsraeliSearch/1.0
Host: dylan.ius.cs.cmu.edu
Email: etamar@xpert.co
JavaBee
Agent: JavaBee
Host: *
Email: info@objectbox.com
JBot Java Web Robot
Agent: JBot (but can be changed by the user)
Host: *
Email: daniel@matuschek.net
JCrawler
Agent: JCrawler/0.2
Host: db.netimages.com
Email: snowhare@netimages.com
AskJeeves
Agent: Mozilla/2.0 (compatible; Ask Jeeves/Teoma)
Host: ez*.directhit.com
Email: postmaster@ask.com
JoBo Java Web Robot
Agent: JoBo (can be modified by the user)
Host: *
Email: daniel@matuschek.net
Jobot
Agent: Jobot/0.1alpha libwww-perl/4.0
Host: supernova.micrognosis.com
Email: ajack@corp.micrognosis.com
JoeBot
Agent: JoeBot/x.x,
Host:
Email: rwaldin@primenet.com
The Jubii Indexing Robot
Agent: JubiiRobot/version#
Host: any host in the cybernet.dk domain
Email: jakob@jubii.dk
JumpStation
Agent: jumpstation
Host: *.stir.ac.uk
Email: j.fletcher@stirling.ac.uk
image.kapsi.net
Agent: image.kapsi.net/1.0
Host: addr-212-50-142-138.suomi.net
Email: Jaakko.Heusala@kapsi.net
Katipo
Agent: Katipo/1.0
Host: *
Email: Michael.Newbery@vuw.ac.nz
KDD-Explorer
Agent: KDD-Explorer/0.1
Host: mlc.kddvw.kcom.or.jp
Email: matsu@lab.kdd.co.jp
Kilroy
Agent: yes
Host: *.oclc.org
Email: kilroy@oclc.org
KO_Yappo_Robot
Agent: KO_Yappo_Robot/1.0.4(http://yappo.com/info/robot.html)
Host: yappo.com,209.25.40.1
Email: office_KO@yappo.com
LabelGrabber
Agent: LabelGrab/1.1
Host: head.w3.org
Email: jamieson@mit.edu
larbin
Agent: larbin (+mail)
Host: *
Email: sebastien.ailleret@inria.fr
legs
Agent: legs
Host:
Email: admin@magportal.com
Link Validator
Agent: Linkidator/0.93
Host: *.mitre.org
Email: tgimon@mitre.org
LinkScan
Agent: LinkScan Server/5.5 | LinkScan Workstation/5.5
Host: *
Email: sales@elsop.com
LinkWalker
Agent: LinkWalker
Host: *.seventwentyfour.com
Email: rbryant@seventwentyfour.com
Lockon
Agent: Lockon/xxxxx
Host: *.hitech.tuis.ac.jp
Email: search@rsch.tuis.ac.jp
logo.gif Crawler
Agent: logo.gif crawler
Host: *.inm.de
Email: sevo@inm.de
Lycos
Agent: Lycos/x.x
Host: fuzine.mt.cs.cmu.edu, lycos.com
Email: fuzzy@cmu.edu
Mac WWWWorm
Agent:
Host:
Email: lemieuse@ERE.UMontreal.CA
Magpie
Agent: Magpie/1.0
Host: *.blueberry.co.uk, 194.70.52.*, 193.131.167.144
Email: Keith.Jones@blueberry.co.uk
marvin/infoseek
Agent: marvin/infoseek (marvin-team@webseek.de)
Host: arthur*.sda.t-online.de
Email: marvin-team@webseek.de
Mattie
Agent: M/3.8
Host: mattie.mcw.aarkayn.org
Email: matt@mcw.aarkayn.org
MediaFox
Agent: MediaFox/x.y
Host: 141.99.*.*
Email: sfx@uni-media.de
MerzScope
Agent: MerzScope
Host: (Client Based)
Email:
NEC-MeshExplorer
Agent: NEC-MeshExplorer
Host: meshsv300.tk.mesh.ad.jp
Email: web-dir@mxa.meshnet.or.jp
MindCrawler
Agent: MindCrawler
Host: *
Email: support@mindpass.com
mnoGoSearch search engine software
Agent: UdmSearch
Host: *
Email: support@mnogosearch.org
moget
Agent: moget/1.0
Host: *.goo.ne.jp
Email: moget@goo.ne.jp
MOMspider
Agent: MOMspider/1.00 libwww-perl/0.40
Host: *
Email: fielding@ics.uci.edu
Monster
Agent: Monster/vX.X.X -$TYPE ($OSTYPE)
Host: wild.stu.neva.ru