Dissecting the New Related Articles Widget - Part 1

Hi friend,


This post's purpose is for dissecting my new Related Articles, so that other people can make the best use of it.

As with anything else in life, when one starts to do something, there must be at least a reason behind, otherwise, why bother? This is especially true in the case of the existing Related Articles, I mean to redesign it, for many shortcoming reasons below:

1. The tool doesn't allow the user to apply CSS if he or she wants to,

2. The returning related posts are only the latest posts of certain labels,

3. There is no paging system,

4. The most relevant returning posts are not always bubbling up to the top, since there is no system to rank which label is more important than the other.

5. The widget is designed as a "blog widget", so there are lots of unwanted code in the tool.


My goal is to create a new widget which:

1. Blogger, Wordpress, and any other web blogging systems, or even HTML sites can be make used of, as long as they have an organized way to identify labels, such as wrapping them around divs with class as "label" or similar names,

2. Utilize the latest and greatest JavaScipt APIs such as the AJAX Search API, the AJAX Feed API, and most of all, the JQuery API. This is, thus, a chance for me to learn new programming idioms.

3. A chance to revisit the search algorithm to see if I can improve the existing JSON callback system.

Then one day, on my way to work, I had an "EUREKA" moment! Why not using Google for their search engine, using the AJAX Search API to query the list of labels as search keywords, then get the resulting list and massage them the way I want.

To my amazement, not only the AJAX Search API can perform the search pretty fast, it also offers a GUI system so I can hide, display, change colors on my result list! It also has an indexing system, so I can navigate the results from one page to another.

After a week or so, I finish this initial widget. I will dissect the code line by line, so you can have an idea of what it does.

Below is the first version of the widget.

 

<!-- Related Articles - Take 5 - Using JQuery & Google Search API - by Hoctro 8/2010 -->
<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.3/jquery.min.js" type="text/javascript"></script>
<script src="http://www.google.com/jsapi" type="text/javascript"></script>
<script language="Javascript" type="text/javascript">//<![CDATA[

//var website = "hoctro.blogspot.com";
var website = getBaseURL();
var hoctroRelated = "site:" + website + " ";
var labelArray = new Array();

function getParams() {
  $('div.post-footer-line span.post-labels a').each(function(idx, el) {
    var test = 0;
    var textLabel = $(el).html();
    for (var i = 0; i != labelArray.length; i++) {
      if (labelArray[i] == textLabel.toString()) test = 1;
    }
    if (test == 0) {
      labelArray.push(textLabel.toString());
      hoctroRelated = hoctroRelated + '"' + textLabel.toString() + '" OR ';
    }
  }); // each
}
// Callback main function
$(document).ready(function() {
  getParams() ;
}); // function

function OnLoadRelated() {
  // create a search control
  var searchControl = new google.search.SearchControl();

  // create a draw options object so that we
  // can position the search form root

  var options = new google.search.DrawOptions();
  options.setSearchFormRoot(document.getElementById("searchHoctroRelatedForm"));

  // populate with searcher
  var blogSearch = new google.search.WebSearch();
  blogSearch.setUserDefinedLabel("Related Articles");
  var options2 = new google.search.SearcherOptions();
  options2.setExpandMode(google.search.SearchControl.EXPAND_MODE_OPEN);

  searchControl.setResultSetSize(google.search.Search.LARGE_RESULTSET);
  searchControl.addSearcher(blogSearch, options2);
  searchControl.draw(document.getElementById("searchHoctroRelatedResults"), options);
  getParams();
  searchControl.execute(hoctroRelated);
}

// Copied from http://www.gotknowhow.com/articles/how-to-get-the-base-url-with-javascript
function getBaseURL() {
  var url = location.href; // entire url including querystring - also: window.location.href;
  var baseURL = url.substring(0, url.indexOf('/', 14));
  if (baseURL.indexOf('http://localhost') != -1) {
  // Base Url for localhost
    var url = location.href; // window.location.href;
    var pathname = location.pathname; // window.location.pathname;
    var index1 = url.indexOf(pathname);
    var index2 = url.indexOf("/", index1 + 1);
    var baseLocalUrl = url.substr(0, index2);
    return baseLocalUrl + "/";
  }
  else {
  // Root Url for domain name
  return baseURL + "/";
  }
}



  // Important, this has to be called here, otherwise main var is not yet created
  google.load('search', '1.0');
  google.setOnLoadCallback(OnLoadRelated, true);

//]]>
</script>

<style>
  //#searchHoctroRelatedResults a.gs-title, a.gs-title * { color : brown; }
  //#searchHoctroRelatedResults .gs-webResult .gs-snippet { display : none; }
  //#searchHoctroRelatedResults .gs-webResult .gs-visibleUrl { display : none; }
  //#searchHoctroRelatedForm { display : none; }
</style>

<div id="searchHoctroRelatedResults">Loading...</div>
<div id="searchHoctroRelatedForm">Loading...</div>
<h6>Related Articles Widget by <u><a href='http://hoctro.blogspot.com'>Hoctro</a></u></h6>

Let's start with the first few lines:

01 : <!-- Related Articles - Take 5 - Using JQuery & Google Search API - by Hoctro 8/2010 -->
02:  <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.3/jquery.min.js" type="text/javascript"></script>
03:  <script src="http://www.google.com/jsapi" type="text/javascript"></script>
04: <script language="Javascript" type="text/javascript">//<![CDATA[

05:  //var website = "hoctro.blogspot.com";
06:  var website = getBaseURL();
07:  var hoctroRelated = "site:" + website + " ";
08:  var labelArray = new Array();

Line 2: I'm telling the page to load JQuery, version 1.3. This is such an exciting JavaScript API to learn, I'm just a novice learner of this system, but I like it so much already, I will explain some of the code using JQuery later.

Line 3: I'm telling the page to load the Google's loading utility, so at a later time I can load the AJAX Search API.

Line 5: I can use the variable website to directly as "hoctro.blogspot.com", or I can use a function in line 6 (var website = getBaseURL();) I copied from the web, which will detect the page URL, then strip the page and only contains the root. This is important because at a later time in my search, I use this blog name to restrict Google to give back results only within my blog.

Line 6: since I comment out line 5, line 6 will hold the text containing my root (http://hoctro.blogspot.com/)

If you want to save room by key in your web root in line 5, then you can uncomment line 5, get rid of line 6 and the function getBaseURL() altogether.

My goal was to let the novice user to just cut and paste the code to the HTML widget, without any JavaSript knowledge, hence line 6.

Line7: var hoctroRelated = "site:" + website + " ";  This variable will hold the searching text to send to Google. Right now it is "site:http://hoctro.blogspot.com/ "

One modification to this hack is available already! If you want Google to search the entire WWW, then you need to modify the line to become:

var hoctroRelated = " ";

Line 8: stores the array of currently found labels.

Next, we will inspect the function getParams(). This is the function I use to extract labels out of the page to form the query text.

09: function getParams() {
10:  $('div.post-footer-line span.post-labels a').each(function(idx, el) {
11:    var test = 0;
12:    var textLabel = $(el).html();
13:    for (var i = 0; i != labelArray.length; i++) {
14:     if (labelArray[i] == textLabel.toString()) test = 1;
15:    }
16:    if (test == 0) {
17:      labelArray.push(textLabel.toString());
18:      hoctroRelated = hoctroRelated + '"' + textLabel.toString() + '" OR ';
19:    }
20:  }); // each
21: }

Line 9: Originally, I let the function to stay inside the $(document).ready(function() JQuery get-ready function, but I found out that I need to sub it out to call it twice, since the AJAX callback and the JQuery ready are not always in synch. As a result, my search text sometimes has nothing in it, which is kind of an embarassment. Now I think the problem is solved by calling the function getParams() one more time, right before sending the search.

Line 10: This is JQuery in action! There is so much to explain in this one line. Basically, I tell JQuery to search for all the divs with the class post-footer-line, then all the span tags inside them with the class post-labels, then finally grab all the a tags! For each of the resulting a tag it finds, do the things inside the unnamed function inside line 11 to 19.

This line is so important, for it allows any other blogging system or regular web pages to look for the labels/ categories unique to their naming convention, and to grab them.

In other words, I use JQuery to stay away from using the blogger's b:loop for collecting labels. Now the widget is no longer depending on Blogger template code.

It is so important then, that you must show the labels in your post(s). Do not hide them in the footer area.

Line 11: this var is to detect if a label is already on the array. If it's equal 1, then the label is already stored in the labelArray.

Line 12: the textLabel stores the content of the a tag, in this case the label.

Line 13 to 15: This is a test to see if the label text (I used toString() function to be 100% sure to get text) is already existed in the array. If so, test = 1 

Line 16: if the variable is still zero, with is the default assignment value, then it means this is a new label, I need to add to the search.

Line 17: but before that, I need to add the label to the list of found ones, so the next "each" round it can be used to compared against the next found label.

Line 18:  hoctroRelated = hoctroRelated + '"' + textLabel.toString() + '" OR '; after many trials and errors, I find out that the label need to be in double quote, and I need to use the OR operator to make sure the result can search on all the keywords/labels. Then, if not found, Google can also use each label as a search term.

You can play with this line to get different results from google based on your need, if you want to.

An example of the search would be like the pic below:

site:http://hoctro.blogspot.com/ "Related Articles" OR "Widgets" OR "Blogger Tools" OR "Miscellaneous" OR "Template Designer" OR

I could easily use a string concatenation function to remove the last OR, but there's no need to, the search understands it ...


The last three lines are just for closing the if, the unnamed each function, and the getParams() function, respectively.


To be continued in part 2 ...