Question : Applying Weight Values to Search Engine

Hello, I would like advice about the following.

I have made a crawler in python which can analyze the following weights to apply to a page.

(1.0,self.locationscore(rows) - location of keyword on page; top is better than bottom
(1.0,self.frequencyscore(rows) - how ofter keyword appears
(1.0,self.pagerankscore(rows) - PR(A) = 0.15 + 0.85 * ( PR(B)/links(B) + PR(C)/links(C) + PR(D)/links(D) )
(1.0,self.distancescore(rows) - how close keywords are together
(1.0,self.inboundlinkscore(rows) - number of inbound links
(1.0,self.linktextscore(rows,wordids) - determine the value of the links
 value = 6

In your professional opinion what are good values for each?  

** Notes **
 - I would like to base the results out of 100
 - I have considered altering the values dynamically depending on the search critera

Answer : Applying Weight Values to Search Engine

UltraFlux -

That is a very interesting question.

The weight values you will get from arbitrary experts will be wide ranging and unlikely to produce any useful result. However there is a much better way.

First you must construct a training set of data. This would be the SERPs for a range of keywords.  Then using a hundred or so random weightings you apply your formula to the training data set. You then rank the formula according to how well they predict the position in the SERPs.

Next you take the top fifty and mate them with the one below in the ranking order i.e. take half of one and add it to half of the other to produce a new formula.

To the fifty new formula you then add fifty new random formula and then again you apply your formula to the training data set, rank and then mate. Repeat for many iterations until the top few formula become stable.

Then you can use the final formula and see if it is as good in the wild as it is with your training set.

There is probably no need to write your own software to do all this because there are quite a few genetic algorithm scripts and software available already.

I would be interested to know the result when you have finished. (Email in Member Profile)

- duz
(On a rare visit).