[drupal-support] Clean URL's and Spiders

[Ross Kendall]

I think that the problem might be that the spider does not recognise 
<base> url, and so it get's all the links wrong.

This is the spider's problem (it should recognise <base> tag)

I'm sure the spiders for all the major search engines would get this right.

Since you didn't explain very clearly, I think Gerhard was guessing at 
your problem (just like I am :)

Basically, I don't think you have too much to worry about.

Cheers,
Ross.

Thatguywhowon'tgivehisname wrote:

>Not sure what you mean Gerhard.
>
>
>--- Gerhard Killesreiter
><killesreiter at physik.uni-freiburg.de> wrote:
>
>  
>
>>On Sat, 19 Feb 2005, Thatguywhowon'tgivehisname
>>wrote:
>>
>>    
>>
>>>I took a look at
>>>
>>>      
>>>
>http://www.searchengineworld.com/cgi-bin/sim_spider.cgi
>  
>
>>>the other day and noticed that when
>>>      
>>>
>>spider-checking
>>    
>>
>>>any drupal-based site that has enabled clean url
>>>functionality the spider doesn't fill in the
>>>      
>>>
>>domain
>>    
>>
>>>name.
>>>
>>>Give it a try and see what I mean.  I am not that
>>>worried since my pages show up in Google and
>>>      
>>>
>>Yahoo.
>>
>>'Spider title' you mean? It is set for my site. I
>>think it takes the title
>>tag for the site you are looking at. Maybe your
>>theme doesn't set it.
>>
>>Cheers,
>>	Gerhard
>>-- 
>>[ Drupal support list | http://lists.drupal.org/ ]
>>
>>    
>>
>
>
>
>		
>__________________________________ 
>Do you Yahoo!? 
>The all-new My Yahoo! - Get yours free! 
>http://my.yahoo.com 
> 
>
>  
>