Google's Weakness, AltaVista's Strength 326
Some people love the results they get at Google, others are often disappointed. To a large extent, both the pluses and the minuses derive from Google's ranking system, which (as the folks at Google explain www.google.com/technology) depends largely on the number links to a particular page and the relevance of the content on those linking pages to the content on the target page, and the quality of the pages doing the linking.
Thanks to that complex and brilliant system, over time, the best pages often rise to the top of search lists. But that takes time -- a lot of time.
It works great for old, established sites to which many other old, established sites have linked. (It works great for my site :-) www.samizdat.com ). But new sites, regardless of the quality of their content, get short shrift. It takes 2-3 months for the new pages to get into the Google index. Then it takes time -- perhaps years -- for other "important" sites to discover the new site and link to it; and then months more for the new versions of those pages with those new links to get into the Google index.
So if I'm looking for content that is likely to have been on the Internet for a year or more, Google is great. But if I'm looking for fresh content, I'll go elsewhere.
For me, for years "elsewhere" meant AltaVista -- for two reasons. AltaVista used to add new pages to its index, for free, within two days of submission, while other search engines typically took weeks or even months. That meant they had the freshest content. In addition, AltaVista provided you with a set of very precise commands that couldn't be matched anywhere else.
Over the last year, as AltaVista has struggled to become profitable, they have destroyed their beautiful free submission process, trying to force Web sites to pay for submission. Free submissions (which typically come from the kinds of content-rich sites that I'm interested in) now seem to take three months or more -- no better than the other search engines and often worse.
Fortunately, the powerful commands remain -- for instance, the ability to exclude as well as include terms in your query. AltaVista lets you use minus signs and plus signs to indicate what you really don't want and what you do want. And for some specialized searches the exclusion is essential.
For instance, say you want to know what Web pages outside of your own site have links to your pages. At Google, I can do a search for link:samizdat.com or get the same results by going to their "Advanced" search and using their "page specific search" to find pages that link to a particular page. But my results are then littered with pages from my own site -- information I don't need and don't want. At AltaVista, I can search for +link:samizdat.com -host:samizdat.com and get exactly what I want -- finding out who thinks enough of my pages to have linked to me without my having contacted them: a valuable list of well-wishers and potential partners.
Similarly, Google lets me restrict a search to a particular Web site. For instance, if I include in my query the term site:samizdat.com or in Advanced search under Domains I choose to restrict the search to that domain, Yes, I get results only from that site. But to use that command, I need to have additional query terms: site:samizdat.com alone generates no results.
At AltaVista, however, I can search for host:samizdat.com and get a complete list of all the pages at my site that are in the AltaVista index. Or I can search for url:samizdat.com/isyn and get a list of all the pages in that directory at my site are in the AltaVista index. Or I can search for url:samizdat.com/consult.html to see if that particular page is in the index.
In other words, AltaVista provides a higher level of precision and the ability to get information that is particularly valuable to people in charge of Web sites and Web-based marketing projects. And if they'd just fix their free submission process and provide the service they used to, they'd kick Google's ass for searches for current information.
P.S. -- The folks at Google are very proud that their system defies human tampering. In fact, what they've done is encouraged the development of bizarre business models structured to take advantage of their link-based ranking system. For instance, Webseed Publishing now has over 1000 sites, all with different domain names. These content-rich sites are each run by different dedicated individuals. (I'm one of them :-) In many cases, the content deserves high rankings for its quality. You might wonder why the umbrella business for all these sites bothers to maintain over a thousand different domain names, when it would be far simpler and cheaper to have them as directories under a single domain. But because the domains are different, the many thousands of links these sites have to one another all count toward the automated calculation of their popularity and quality at Google, giving them all a boost in the rankings and hence bringing Webseed more traffic and hence more revenue.
P.P.S. -- AltaVista appears to be making a comeback. Six years ago, when I was in the Internet Business Group at Digital and Digital owned AltaVista, about a third of the traffic to my Web site came by way of AltaVista. Whenever AltaVista had a glitch, I saw it immediately in my traffic stats. In fact, I sometimes was able to alert the engineers at AltaVista about problems before they had noticed them themselves. Over the years, due to increased competition from other search engines and also due to the business folks at AltaVista making bad decisions and jettisoning great capabilities/services (like 2-day free submissions, their affiliate program, LiveTopics, and newsgroup search), the number of people finding my pages by way of AltaVista plummeted. By January 2002, only 1% of my traffic was coming by way of AltaVista, despite the fact that as a long-standing fan and also as co-author of the book The AltaVista Search Revolution, I had lots of information about AltaVista at my site. I was actually getting twice as much traffic from the International Atomic Energy Agency (part of the UN), when I had no information at all related to atomic energy. But in recent weeks the traffic from AltaVista has climbed sharply. It now amounts to 6% of my total. I wish I knew why that was happening. In any case, I hope that trend continues.
Wow (Score:3, Interesting)
Re:Wow (Score:4, Interesting)
If the site is unique to its topic then it will appear higher in the rankings immediately as opposed to *yet another PHP site* which might never climb higher than number 80,991. This is not necessarily harmful to the surfers though the owners of the site will not be pleased.
If it's taking your sites a long time to show up in the rankings then chances are it's not a Google problem so much as well.. Is your site really that unique afterall? Are you using the same search terms that the average user looking for your site is going to use? If you're a shoe store in Massachusetts your customers wouldn't find you by searching for shoes- they'd find you by searching for "Shoes" and "MA".
I'm always finding new content with Google, but I never use it to find up-to-the-minute stuff. I never use *any* search engine to find that. I ask myself what it is I want to know and go to a news site related to that item. Chances are that NO ONE has it indexed yet. Not Google, not Altavista.
Isn't that what everyone does?
-Sara
Re:Wow (Score:3, Interesting)
Some who are more familiar with eastern and Australian news might know that a few days ago a young Melbourne couple were detained in China, and sent back to Australia for unfolding a banner. The story isn't important here, what is, is that I went to school with the girl, actually that's not important either, what is important is that I plugged Emma Dodrell (the young lass's name) into google _that night_, less than 12 hours on, and got 4 related articles from news sites around the world.
Somewhere the gears are churning.
Re:Wow (Score:2, Funny)
"I'll just pull up a search on visivimo... I mean vimisivio.... I mean vivimiso.... I mean.... awww, screw it. Google...."
Let's face it, if you're going to try to overthrow a behemoth like Google, you should start by using a simple, easily-remembered, not-pointlessly-unintelligible name. Vivisimo? Teoma? Not bloody likely.
a lament for text-only altavista (Score:2, Interesting)
powerful commands and no ads... what a concept!
i only switched to google after altavista finally got rid of their text-only page.
Re:a lament for text-only altavista (Score:3, Insightful)
Check out www.raging.com [raging.com]. Altavista search; nothing but text.
They still have text-only (Score:3, Informative)
All I know is this... (Score:2, Insightful)
I use both of them as well. (Score:5, Insightful)
For me, Google doesn't have to be the perfect search engine - it's already enough. I type in google.com and it loads damn near instantly. There's no annoying advertisements, and I can search in h4x0r or Sveedish Chef, bork bork bork.
If I can't find what I want on Google, fine, I'll use another engine. And what's wrong with that? We honestly can't have too many search engines (Well, business problems aside), because each one ends up with different ranking systems, different data pulled up from queries, etc.
Re:I use both of them as well. (Score:4, Interesting)
The Google folks were at a local user group meeting a few months ago. They told us that they have byte counters -- the human kind -- monitoring how many bytes each page served takes. Their mission is to keep the count down.
They got very noisy applause for that statement.
So waitaminute ... (Score:5, Insightful)
Actually, there is lots of good information you provide on the capabilities of search engines. I, for one, would love to see more "A is good for this, B is good for this", instead of simply grouping and competing A & B, suggesting that one can only use one.
IMHO, this is where (free) web services really rule - I can't buy 5 different cars for 5 different reasons I use cars, but in the case of these types of services, the cost of using and switching between these services is very next-to-nil. Hopefully, web services will start encouraging companies to share again, as Google and Altavista may very well demonstrate that sharing market segments with other players makes everyone happier in the long run.
Re:So waitaminute ... (Score:2, Insightful)
A is best for... (Score:2, Informative)
2-3 Months for Google? (Score:5, Insightful)
Now it took months to get into DMOZ, but we did. Yahoo - still hasn't accepted us into our proepr catagory even after 2 or 3 tries over a year and a half.
I think Google could benefit by adding some more advanced filtering command slike Altavista has - I agree they are nice. But the bottom line is, for obscure sites, once you get in Google, look out. Months later we finally got into the other mainstream search indexes (we submitted to them all at the same time) and in teh end Google is THE place for referrals. By orders of magnitude. YMMV, but it seems the other search indexes blew it when tehy killed free submits since folks knwo that they will only return paid sites (plus rank skewring, for $$$, etc)
Only time will tell, but I use Google daily and am happy with the results and performance - no other search engine comes close IMHO
Re:2-3 Months for Google? (Score:2, Informative)
As for new sites, it's been taking a week or so recently. Usually if I don't see it in a week, I head over to their add URL page and submit it.
Talking about Google only using ratings (via number of links) is simplistic. Their index/search algorithms are obviously much more complex than that, and appear to utilitize a wide range of methods beyond simply rating.
Re:2-3 Months for Google? (Score:2, Informative)
Consequently, if there is a link to a new page appearing on the index page, and you happen to have (very) high PageRank, the new page might get indexed as well, outside the 4-week frame.
Re:2-3 Months for Google? (Score:5, Informative)
I suspect this is due to a more frequent crawl at sites Google considers interesting, so if you put up a site and no one hears about it for a while, it could take longer, but in general they're quite responsive.
Re:2-3 Months for Google? (Score:2, Interesting)
The first 6 links were less than 12 hours old. So maybe "Months" should be understood as "Hours".
I also used to use Altavista.
A very long time ago...
I'm sticking with Google... for now (Score:3, Insightful)
The one thing I've noticed about these "flaws" in Google "exposed" on
google toolbar makes them the obvious choice. (Score:5, Interesting)
on all my windows boxes it is one of the first things i install.
google is probably the best search tool right now, and they make using it a breeze. altavista used to be the best search tool, but they made it harder and harder to use, and then search tool lost its top spot. totally different situation. if google looses its top spot in the search tool field, i'll still use it for its ease of use.
Re:google toolbar makes them the obvious choice. (Score:3, Informative)
You can add your own toolbars for any search engine. I have several samples for Mozilla [mozilla.org] on my webpage [barsoom.org]. I also include a very brief description on how to add other search engines, and/or add them to IE.
-- Agthorr
Not only that... (Score:5, Informative)
Also, the coolest feature of the Google toolbar IMHO is not even the instant search, but the "Highlight" button. Gone forever is hitting Ctrl-F and typing in a search term. Just search for something in Google, go to a result, and hit "Highlight" -- the search terms are instantly highlighted. This saves me an incredible amount of time when I'm searching through, say, mailing list archives.
The Google toolbar is one of the biggest reasons I use IE. (Well, that and the fact that page developers, including myself, follow the rule of thumb "Design so that it looks good in IE and works in Netscape.") But anyway, I digress. If you're using IE, check out toolbar.google.com [google.com] and download it.
Re:Not only that... (Score:2)
I still use the Google toolbar for searching within a page though. Somehow, the toolbar picks up the search terms when the search page is loaded.
Another cool feature is that you don't even have to perform a search to be able to use the highlight and search features. Just type the strings into the Toolbar and the search buttons appear right away. You can then search within the current page.
Finally, you can press Alt+G to bring the cursor into the toolbar. Pressing Alt+Enter instead of Enter in the toolbar is the equivalent of the Feeling Lucky feature.
Re:Not only that... (Score:3, Interesting)
Andrea
Re:Not only that... (Score:2)
Please enlighten me re: "tabbed browsing"... (Score:2)
Check out this screenshot [corkyserver.com] that I did for my friend, and tell me how this is different from tabbed browsing in Mozilla.
Re:Please enlighten me re: "tabbed browsing"... (Score:2)
Forget IE, forget Mozilla, and though Opera's tabs are decent, check out Netcaptor [netcaptor.com]. Based on IE's component engine, it does justice to tabs.
And, to stay on topic, you can configure QuickSearches with almost EVERY search engine (google included) where you just type an abbreviation and the search term in the address bar. I, for instance, have Google as g, AltaVista as av, Network Solution's Whois as wh, etc. It'll change the way you browse.
Note, that it's shareware, but it's really worth the money. Up to around version 6.5 it was freeware, then they included ads (yech!), then went to a 30-day demo. But at $29.95, it's definitely worth it.
Re:Please enlighten me re: "tabbed browsing"... (Score:2)
Thanks :) (Score:2)
Thanks again.
Re:google toolbar makes them the obvious choice. (Score:2)
check out lsdie (Score:3, Funny)
(i wrote it)
it's for ie, it installs a little bar like the IE Search bar but a lot smaller, and no ads, and has support for selecting among about 20 different search engines, easily selectable in a convenient interface....
Re:check out lsdie (Score:2)
Googlizer (Score:2)
It takes the current clipboard, and opens a Google search for it. Small tweaking (it's only a short amount of code) would, I'm sure, make it work for other sites. Telsa claims it as working with Gnome, but I use IceWM, and it works great. A real boon, and no wasted space in my netscape toolbar.
Re:google toolbar makes them the obvious choice. (Score:2)
Faster and Faster (Score:5, Insightful)
Yes, it's true that Google's algorithm prevents new content from being ranked high, because no one has linked to it yet necessarily, but that's by design - it is indeed at that point unproven in terms of quality. However, the spidering process can use improvement so that when many many people link to this new site just a few days later, it now ranks higher.
Google specifically mentions (in previous interviews I read with employees) that they're always working on updating the speed, as well as the precision. The longterm goal is to significantly decrease the amount of time it takes to respider everything, and therefore make the info more relevant faster. I trust that they will continue to improve, and eventually this differentiation between "Altavista is better for new stuff, Google for old" will go away completely.
I love AltaVista (Score:4, Interesting)
Re:I love AltaVista (Score:2)
Google is slightly better, but still not good (Score:3, Interesting)
Of course you can find things with search engines now. Google's "trick" of counting links helps a little bit for a particular class of query, which is when you know the name of an organization and you want to find its site...it works well because more people will link to the site as opposed to other sites that discuss it. But as I have written elsewhere, if AltaVista is 99% lame, then maybe Google is only 97% lame...which is three times better, but still terrible if you take a step back.
Now Google is doing a lot of good things outside from its basic search engine, which should be applauded. The caches, saving old Usenet posts, the image and catalog searches, etc. are all good things -- but they don't affect its basic ability to search well.
Further karma ho' expounding can be found right here [osopinion.com].
- adam
Neat commands! (Score:5, Funny)
+link:mysite.com -host:mysite.com
to see how many people have linked to them
Google is still my first pick (Score:4, Insightful)
No tool is the best tool for every purpose and perhaps many people should give other search engines a try and see the strengths.
However, I don't really see that point of an article that is simply a Hoorah for one service over another with differing models of profit and aims.
The author had simply pointed out that AltaVista as opposed to other search engines has advanced searching abilities including the ability to exclude terms. No, it has to be an AltaVista over Google article.
Different tools for different times and different uses.
_______________________________________________
luck (Score:5, Funny)
Re:luck (Score:3, Funny)
Then technically you're breaking Google's TOS. You are supposed to be feeling lucky BEFORE you press the button.
Google has power features too. (Score:5, Insightful)
See http://www.google.com/help/refinesearch.html
Altavista has a long way to go (Score:2, Interesting)
Back then you could submit to Altavista, and have a good ranking within a week.
Over time, the relevance of the returned results dropped dramatically and the time to get a site listed plummetted, quite often taking longer than Yahoo!
Then Google came along and I haven't looked back since. I've consistently been able to find the results I'm after thanks to the way Google indexes sites.
I'm now able to almost guarantee clients that their sites, whether old sites that are being revamped or new sites that are freshly hatched, will be ranked well within Google and also ranked within a short period of time. I think the longest I've ever had to wait for a site to be fully indexed is three months.
Plus the indexing of database generated pages and PDF documents by Google is a life saver. Without this feature a lot of the content I develop would be lost.
I think it will take a miracle to get Altavista back on track. I wish it was as great as it once was, but for now it's relegated to one of the less important engines both from a searching and a submitting point of view.
Sometimes I hate Google... (Score:5, Interesting)
I'm a librarian. It is the most difficult time in history to do library research. There are hundreds of overlapping commercial databases out there, each with their own coverage, interface, and search engines.
Students used to locating information with Google are appalled at the steps it takes to locate a scholarly journal. You need to browse a list of subject databases, search them, then locate a printed copy of the journal via our catalog (a growing but still small percent of journals are available online).
Someday searching the various literary databases may be as easy as Google, but in the meantime there are drastic capitalist impediments to making it easy to do library research.
... so ask a Librarian if you ever need help
Re:Sometimes I hate Google... (Score:2)
What we need to do is be patient a few more years, and the same running-dog capitalistic system (you know, the same evil system that brought us Google) will finish off those inaccessible, elitist journals nicely.
The right tool for the job (Score:2, Insightful)
I have no need for altavista. I don't care if yo use altavista. Google works just fine for me. If altavista works just fine for you, so be it. Use it. No one cares.
All this speculation on the future of google recently is ludicrous. "google bombing" poses no threat. The people who work there are extremely talented. If it becomes a problem, they will undoubtedly fix it.
Google is the most popular search engine in the world, and with good reason. They are not going to give that up.
So will everyone please just sit down, shut up, and stop bickering. Use whatever tool works best for you.
Re:The right tool for the job (Score:2, Interesting)
thing is, i'm sure there are some extremely talented people working at inkotomi, altavista, etc. but, those companies have been around long enough to have to 'fess up to the accountants and justify the work they do.
google, i think, is just hitting that stage - the google competition, whilst being an ingenious idea to most of you guys, suggests to me (cynical engineer type that i am) that they have run stone dry of ideas...
talented people working at inkotomi, altavista, etc. but, those companies have been around long enough to have to 'fess up to the accountants and justify the work they do.
google, i think, is just hitting that stage - the google competition, whilst being an ingenious idea to most of you guys, suggests to me (cynical engineer type that i am) that they have run stone dry of ideas...
Yup Google Rocks, (Score:2, Funny)
The other day
He was going to screw up the spyware system by searching google for "CROSSDRESSING MONKEY PORNO" a bunch. I replied with a physical link for search google for this [slashdot.org]. Sometime later an anonymous coward posted that the
But these posts on
-Sean
Re:Yup Google Rocks, (Score:2, Funny)
(go ahead, mod me down just for bringing up that site..)
Did you even read Google's tip pages... (Score:2, Interesting)
Abuse of Google's service. (Score:2, Interesting)
The way I'm interpreting that is abuse of Google's ranking system. Its an inherently dishonest business practice and I'm led to the conclusion that (Webseed Publishing && affiliates)==dicks.
Copernic (Score:2, Informative)
I have speculated on this problem for some time... (Score:4, Interesting)
The idea is pretty simplistic, although the implementation is complex.
Any communication takes place by translating an idea into a sensory input form.
Examples: Sight (written language, video, sign-language), Touch (brail, texture), Sound (conversation, music), Taste (Like water for chocolate?), Smell (pheromones?).
Obviously, not all of these mediums are easy to work with, but we can certainly start with written language.
All languages use the same basic principle: convey relevant information about a central subject. How they go about doing it is different even between versions of the same language (British English vs. American English).
If we described an objective hierarchy of physical objects described by pure mathematics and implanted them into a central, world-wide database then open-source parsers for each language could handle the task of translating any written text, in any supported language, into this common language. If correctly implemented a search engine could enter into a short dialogue with a person performing a search and then return information very specifically relevant to what the user was searching for.
Example dialogue:
[user]I want information on Mary Jane Carpenter.
[google]There is a very famous person by that name. Her official website is [here]. [Here] is a list of fansites and [here] are some other sites which discuss her. That name is mentioned in [these] sites, but it is unclear if they are talking about the same person. [Here] is a list of other people with that name.
[user]The person I am looking for isn't famous.
[google]Then you are probably looking for one of [these] people.
[user] Are any of those people from St. Lewis?
[google] [Here] is a sight dedicated to a Mary Jane Carpenter from St. Lewis.
This may sound like an impossible streatch but it really isn't. The famous Mary Jane Carpenter has a unique id on her object and many thousands of attributes which uniquely identify it from any other Mary Jane Carpenters. Ambiguity is dictated by the same rules that govern conversation: context.
If I have a page that contains no content other than Mary Jane Carpenter sucks! then a simple fuzzy logic routine should be able to infer that the Mary Jane Carpenter I am talking about is probably the famous one. Other clues could be gained from other parts of my site or other documents which have me as a source.
I realize that I am talking about a HUGE database, but it sure would be handy...
Re:I have speculated on this problem for some time (Score:3, Insightful)
only one thing seperates them for me (Score:3, Interesting)
p.s.
the advanced search page is all text, not even a banner ad so it's almost faster than google to load.
Re:only one thing seperates them for me (Score:2)
Huh? The advanced search page [altavista.com] I see has not only their logo, but a banner ad, and some tables. So not only is it "almost faster" than google to load, it's nowhere close.
Re: (Score:2)
Re: (Score:2)
Google site: workaround (Score:5, Informative)
You can use the following workaround to do a site: search on google without any keywords. Just do "site:yoursite.com -stuff" where stuff is gibberish (bang on the keyboard a bit). For example, this search [google.com] shows 1,290 pages from samizsat.com. On the other hand, an altavista search for that site shows 1,090 hits for pages on that site.
I don't know why Google doesn't allow simultaneous "site:" and "link:" searching, as that is something many users would like to do.
Re:Google site: workaround (Score:2)
Slashdot's new Context Advertisting Scheme (Score:4, Funny)
Features: ICMP echo requests are 37337!
Posted by CmdrTaco on 03:35 PM -- Wednesday March 13 2002
from the leet-nettools-impress-chicks dept.
Hey, Slashdotters! I just found this 37337 tool called pign. You can use it to send an ICMP echo request to IBM.COM. You just type "ping ibm.com"...
And it pings IBM.COM! Check it out:
>ping ibm.com
Pinging www.ibm.com [129.42.17.99] with 32 bytes of data:
Reply from ibm.com : bytes 32 time 80ms TTL=128
Reply from ibm.com : bytes 32 time 80ms TTL=128
Reply from ibm.com : bytes 32 time 80ms TTL=128
Reply from ibm.com : bytes 32 time 80ms TTL=128
Reply from ibm.com : bytes 32 time 80ms TTL=128
Reply from ibm.com : bytes 32 time 80ms TTL=128
Reply from ibm.com : bytes 32 time 80ms TTL=128
Reply from ibm.com : bytes 32 time 80ms TTL=128
Reply from ibm.com : bytes 32 time 80ms TTL=128
(Read more...)
Seriously -- I'm sure more curious people clicked over to samizdat.com than clicked on any of the other ads on the screen (thinkgeek and ibm for me). Maybe there is something to text ads on community sites (ala kuro5hin)
sm
Re:Slashdot's new Context Advertisting Scheme (Score:2, Funny)
To quote the above post...
37337 tool called pignWell, at least the spelling is realistic for a Slashdot article :)
The best thing Google did (Score:2, Interesting)
fun fact: I also tried to get a proposal started for AltaVista to acquire Google in the summer of '99. Aren't you glad I failed?
Don't see how Alta can be more current (Score:4, Informative)
Now I just don't see how AltaVista can give anyone more current results if their bots are featherbedding.
___
Re:Don't see how Alta can be more current (Score:2)
You're overlooking Google functionality (Score:3, Informative)
Re:You're overlooking Google functionality (Score:2, Informative)
+link:samizdat.com -site:samizdat.com
at Google, I only get 4 results.
Meta Comment to the Editors/timothy malformed link (Score:2, Insightful)
"mailto:seltzer@samizdat.com or http://www.samizdat"
Please fix it.
When it is fixed, please dont fuck up my karma by marking this as redundant.
I would consider subscibing if it would gaurantee proper links and spellchecking.
Re:Meta Comment to the Editors/timothy malformed l (Score:2, Insightful)
Typical user? (Score:2)
Plus, We Can Keep An Eye on the "Enemy"... (Score:3, Informative)
It's alive. [google.com]
Re:Plus, We Can Keep An Eye on the "Enemy"... (Score:2)
The question is: why are they different and in what way are they different!
Re:Plus, We Can Keep An Eye on the "Enemy"... (Score:2, Informative)
Informing AltaVista engineers about problems (Score:5, Informative)
Alas, they can no longer be reached. Their search engine is seriously broken. It picks on a site and hits it hard and repeatedly.
They will make 100,000 requests on a site with only 20,000 static items within 24 hours. On our co-operative co-loacted server, we host around 80 sites, many of which are content rich. When Alta Vista choose to visit just one of them, our total bandwidth usage jumps by an order of magnitude.
We have been unable to get past their front line support, I am not prepared to maintain robots.txt on all of our member's sites just to control their broken robot, so we had no alternative but to block their entire subnet at our firewall.
If anyone has evidence that the AV robot is fixed, I'd be happy to let them back in.
Learn your tools (Score:5, Insightful)
Read the excellent information Google has provided about how the engine works, and use the engine with its inner-workings in mind. When you meet the machine half-way instead of trying to dumb it down for the user, you'll get a hell of a lot more done.
In Google's case, taking half a minute to think about what you're looking for, then tossing in a few related bits of jargon or other words relevant to the context you're after does amazing things. With a little forethought, you can almost always find what you're after and be down to a page with nothing but relevant links with just an extra word or two added as filters.
International Atomic Energy Agency - spam crawler (Score:3, Interesting)
According to http://www.leekillough.com/robots.html [leekillough.com] - iaea.org is commonly used as a fake referrer by spam harvesters.
Google slowness is a myth (Score:5, Informative)
I posted Two Kinds of Order by John Marks [demon.co.uk] on March 11th, and mentioned this to some colleagues who might be interested. I linked to it from a Weblog [blogspot.com] or two [blogspot.com],and Doc Searls [weblogs.com] did too.
Today it is number 1 [google.com] on a search for 'two kinds of order' out of over 2 million, and a search for John Marks [google.com] brings the page up in 5th position, despite there being lots of other John Marks's on the net.
Thats what I call fast (and relevant)
You got your score wrong (Score:2, Informative)
Do you believe in death after life?
Re:You got your score wrong (Score:2)
I saw something like this when playing with Pocket GoogleWhacker - sometimes the score for a word would vary from time to time. Distributed systems are like that. I'm sure it will stabilise.
Will google work so well if this become true...? (Score:3, Interesting)
Remember this post from Slashdot [slashdot.org] ? It is about Macromedia wanting Flash to be used to design the entirety of a site.
So, I don't suppose Google can fetch the URLS inside a Flash file (correct me If I'm wrong), so, if Macromedia's dream become true, how would Google cope with it?
BTW, how any search engine would deal with such a catastrophe?
Cheers.
OT: Flash might be about to commit suicide (Score:2, Informative)
So, I wonder if Flash might implode on the basis of their success in the ad market coupled with all the problems of using Flash to generate your pages, plus the simple fact that almost no Flash site actually delivers anything that's still interesting after the first visit so who'd miss it?
TWW
Indexing and re-indexing (Score:2, Interesting)
This doesn't help you out if you're trying to get your new business noticed, which is something site managers care about desperately. It also doesn't help you find the new business that appeared two weeks ago that might be able to help with your problem. Sadly, it's generally the same business owners who care about that case, too, since in general somebody has already beaten you to the punch with their web site and the customer gets the problem solved, without you.
No, it's not perfect, but it solves the problems of web searchers very, very often. It may be less good for web site owners, but compared to the searchers they are in the minority.
Google is too good, no point to advertising. (Score:2, Interesting)
The other day I played with the Google advertising generator, just to see how much an ad would cost and how it worked, not with any intention of advertising. (Check it out, it's fun.) Anyway, I pretended to be advertising a local special-interest club where I am a member. By the time I had picked the advertising keywords that gave me the ad traffic that I wanted, those very same words typed into the search box brought up the club's web site as the third link on page one.
I would advertise why, exactly?
Popularity-based systems (Score:2)
When rating systems -- including Slashdot's -- increase the visibility of what is already popular, they only serve to reinforce the status quo. What's "cool" stays cool, not even necessarily because the audience is that monotonically unimaginative, but because new and different things are filtered out. If, for example, Microsoft actually managed to produce a solid, reliable, inexpensive, and reasonably licensed piece of software, this is about the last place you'd hear about it. With Google's link-popularity system, websites presenting unpopular or dissenting views are much, much harder to find than knee-jerk me-too reactionary sites. This is no small issue, considering that the benefits of a free society are built, ultimately, upon dissent -- and the ability to spread dissent. This is no less true when the dissent is artistic than when it is political.
Almost a decade ago, I used to laugh at the efforts of old-media companies to transform the web into another form of television. It's a lot harder to laugh now, as the chief gateways to the net for the vast majority of the population use sophisticated software to dumb down the net. Sure, the "other" stuff is still out there, but if you can't find it, it may as well not be.
Missed the point of the original article entirely (Score:4, Informative)
His discussion of web search techniques was to illustrate the nature of the problem these would be omnisicents face. Because the data they collect does not have the richly linked nature of web content, all that these governments government entities will be left with is mountains of meaningless data. They will be stuck using AltaVista like searching and matching techniques.
And we all know how useful Altavista is these days.
-josh
Oh! (Score:2)
Google != old content (Score:2, Insightful)
How do you explain this [google.com], then? It's a standard Google search for the terms 'Andrea', 'Yates' and 'verdict'. The top link is hardly a year old, but rather an extremely recent and relevant link [cnn.com] to CNN's site about the trail verdict.
2 - 3 months? That's assinine (Score:2)
google found us on their own (Score:4, Insightful)
but I have to admite to being very impressed, every month the googlebots come to visit, they don't disrupt the site (the National Library of Australia hit us with a denial of service attack called "Pandora" when they tried to suck down the enitre site in one go, complete with recursing loops), and they rank us very highly (perhaps too highly, there are more authoritative sites in our region, we do more comment).
anyway I suspect the author forgot that most users of search engines aren't website owners hoping to be indexed, but people doing searches.
Sites that have been regularly updated for a couple of years tend to be a better source of information than those slapped up yesterday.
Re:why google is flawed (Score:2, Insightful)
but the higher the pages which link to your page are rated, the higher your page is rated.
this means if you just link from geocities pages which are "bad" rated themself (cause there is no content), links to your page doesn't give you any advantage.
Re:why google is flawed (Score:3, Funny)
Re:why google is flawed (Score:2)
You probably don't even need Geocities to astroturf your site...configure Apache for virtual hosting, grab a bunch of names from dyndns.org [dyndns.org] or whatever, and assign those names to the different "slots" on your server. Depending on how Google is set up, you might not even need a bunch of different names; take advantage of subdomain delegation to create alpha.foo.dyndns.org, bravo.foo.dyndns.org, charlie.foo.dyndns.org, etc. and use those for the massive cross-linking. One link from your site (if Google has already indexed it) into the ratsnest ought to set the whole thing in motion.
The name is better? (Score:2, Insightful)
Are you telling me that if google switched names and interfaces with a terrible search engine (like, say, excite or lycos), you would start using that?
You, sir, are stupid.
Re:The name is better? (Score:2)
And actually yes the name does matter because I could never remember altavista's web address. I used Yahoo for awhile, never liked Lycos. Ask was neat, but the style of results was too messes up for me. They kept on adding more types of results to their page and it just didn't make any sence to me.
Google, has a simple, orginized resultset that (to me) makes sence. And their pages are so simple that they take no time to load, even on an old modem. (unlike ask, or other search engines)
Re:conspiratorial holiday (Score:2, Funny)
Re:MSN is better (Score:2, Interesting)
Ahh, you mean slashdot.jp?
Which happens to be registered to VA Linux Systems Japan, whereas slashdot.org is registered to OSDN, who happens to own VA Linux Systems Japan?
You mean that link? You mean Slashdot Japan?
How ridiculous is that?
Re:MSN is better (Score:2, Interesting)
Re:Just out of curiosity... (Score:2)
Hmmm. I think this question would make a good new sig.
Re:Plus Minus search terms? (Score:3, Interesting)
In AV's simple syntax, unadorned terms are not required to be in the results, they are only preferred. If you want to require a term, you must prepend a plus sign. Google does not allow you to distinguish between required and preferred terms. This may arguably be simpler, but it takes away control from you, the user.
AV also offers a separate advanced syntax which provides support for nested booleans, and positional operators like "near", "within", "before", and "after". Google, while it allows a single level of simple booleans, does not provide any means by which to nest them. It completely lacks positional operators other than phrase matching. Again this takes control away from you, the user.
It never fails to mystify me why Slashdot readers, a crowd biased strongly towards programmers, engineers, and Unix users, namely people who love to have lots of control over things, would favor a dumbed down search service! (I agree that there are other problems with AltaVista, such as the annoying popunder ads. However, these have nothing to do with the quality of its search syntax.)