CrawlTrack: Tips and Techniques for Webmasters

If you’re looking for a web analytics program but don’t want to use Google Analytics, you might want to give CrawlTrack a look. This web application goes beyond simple analytics to help secure your website. Let’s take a closer look.

CrawlTrack is a web analytics and security web application for PHP-MySQL powered websites. It is similar to Google Analytics in the way that it tracks visitors and traffic sources. However, one of the positive aspects of this application is the ability to block website hacking attempts. Aside from offering traffic analysis/monitoring and security benefits, it is open source, which means you can implement it in your website for free.

One of the most serious security problems on the Internet is website hacking. Unfortunately, a lot of web applications are still unsecured and do not offer anti-hacking protection. CrawlTrack has anti-hacking features that detect two of the most common hacking methods, namely MySQL injection and code/JavaScript injection. 

This is a two-part tutorial. The first part deals with the concepts, installation steps and integration of CrawlTrack into your website. The second part will discuss how to maximize the use of CrawlTrack, especially for interpreting and understanding website statistics, as well as maximizing the security of your website using their security features.

Unlike most common web analytics scripts that use JavaScript (such as Stat Counter, Google Analytics and Site Meter), CrawlTrack is an entirely PHP-based script application. This means that the public cannot see CrawlTrack’s code in the browser, because it is implemented on the server side. This is advantageous in terms of security.

 

There are basically three major application types for CrawlTrack, namely Public Flow, Admin Flow and Security Flow.

“Public flow” is a method of tracking employed by CrawlTrack when monitoring and gathering public web statistics data. By embedding the code (to be discussed later), this application can track and monitor user statistics for that page; all of the data will be stored in your website MySQL database.

“Admin Flow” is a method of browsing important web statistics data which is done by the website administrator. The administrator needs to log-in in order to view important web related statistics. Once log-in, the crawl track will then communicate (vice versa) to the MySQL database to show statistics in the browser in terms of graphs and numerical figures.

“Security Flow” is a method employed by CrawlTrack in detecting and blocking hacking attempts. However, note that it can only detect and block hacking attempts in pages/URLs that are embedded with CrawlTrack PHP code. CrawlTrack employs a real time method of detecting hacking by means of user input profiling (analyzing user inputs and intentions) and then blocks any hacking attempts if set.

 

Also, bear in mind that CrawlTrack is not a complete anti-hacking solution. You are still required to keep your CMS/web application up to date, to make sure it is not vulnerable to any security exploits.

Before installation, it is important to find out whether your website is compatible with CrawlTrack. Below are the requirements:

  • Apache as the web server (not strictly required, but recommended).
  • A minimum of PHP version 4.3 for server side scripting (required)/Dynamic PHP template-based.
  • MySQL database (required). 

Once you have met these requirements, follow the steps below:

Step 1. Download CrawlTrack here: http://www.crawltrack.net/download.php

Step 2. Once the zip file has been downloaded, unzip it.

Step 3. Prepare the file for uploading. Make sure the folder is two levels (crawltrack3-1-2 à crawltrack3-1-2 files). After unzipping, it is a three-level file structure: crawltrack3-1-2 à crawltrack3-1-2 à crawltrack3-1-2; you need to make sure it is a two-level folder. See the example below for how to make this a two-level structure:

 

Step 4. Open your favorite FTP client program and upload the folder to the root directory of your website.

Step 5. It is highly recommended to have a separate database for CrawlTrack. However, this is not required, and if your web host only allows you to use one MySQL database, this is still possible for CrawlTrack. Make sure you have the following database permissions/information:

  • MySQL username
  • MySQL password
  • MySQL hostname
  • MySQL database name

Step 6. Open your favorite browser, and in the address bar, type the path of the CrawlTrack folder. For example, if the name of the folder is “crawltrack3-1-2” and it is uploaded to the root directory of your website: http://www.yourwebsite.com , the path to be entered in the browser address bar will be: http://www.yourwebsite.com/crawltrack3-1-2. Then press the enter key.

Step 7. The Crawl track installation screen will then pop up. Select the appropriate language.

Step 8. You are then required to enter database information based on what you have prepared earlier. Enter it:

User MySQL à Your MySQL username

Password MySQL  à Your MySQL password

Host MySQL à Your MySQL hostname

Database MySQL à The name of your database

Step 9. Enter the website name. You can use the domain name. For example, if your domain is www.php-developer.org, enter “PHP Developer”

Step 10. In the site URL, enter the domain name without “http://” at the beginning and “/” at the end. So it will just be:

www.php-developer.org

Step 11. You need to set up your account. Enter your desired login and password.

Step 12. The installation is now complete; you can log in.

Step 13. You need to get the embed PHP code which you will need to integrate with your website. After logging in, go to your CrawlTrack administration page and click the icon with the “wrench” symbol:

 

In the administration page, click “Create tags to insert on your website.” When you see your website name, click “OK.” Use the standard tag; for example if your website domain name: www.thisisyourwebsite.com , below is what the standard tag looks like:

$crawltsite=1;

require_once("/home/www/thisisyourwebsite.com/crawltrack3-1-1/crawltrack.php"); 

It is possible to use one CrawlTrack installation for different domains; this will be covered in the second part.

Step 14. You need to embed/integrate CrawlTrack into your website’s PHP- powered template to make it fully functional.

If you are using WordPress, find the footer.php file in your current template. Open it and then paste in the code. In the footer just before </body> is where you see most analytics code, like Google Analytics; embed the code below it:

<script type="text/javascript">

var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www.");

document.write(unescape("%3Cscript src=’" + gaJsHost + "google-analytics.com/ga.js’ type=’text/javascript’%3E%3C/script%3E"));

</script>

<script type="text/javascript">

try {

var pageTracker = _gat._getTracker("UA-xxxxx");

pageTracker._trackPageview();

} catch(err) {}</script>

<?php

$crawltsite=1;

require_once("/home/www/thisisyourwebsite.com/crawltrack3-1-1/crawltrack.php");

?>

</body>

</html>

 

Above is an example of how to integrate CrawlTrack code into your template. Make sure that the embed code is within the PHP tags (<?php and ?> ) and the file name uses the .php extension, or it won’t work.

In some rare cases, it is possible to use CrawlTrack in .htm. For details, refer to the CrawlTrack documentation:

http://www.crawltrack.net/nonphppage.php

There are special cases in which CrawlTrack introduces some kind of incompatibility with the templates. For example, the original documentation states to install the tag on the index.php file at WordPress, but doing this will cause template distortion in sophisticated templates like Arras Theme. In this case, use another generalized PHP template file that is used by all URLs (in this case, footer.php).

You can even test to see if CrawlTrack is correctly integrated into your website by going to CrawlTrack’s administration page, clicking on “Create a test crawler,” and then clicking that button to create a crawler. Follow the instructions on that page, and then open a new tag in your browser. Type in the home page URL of your domain.

After the home page is completely loaded on the browser, go to your CrawlTrack dashboard, refresh it, and then click “Crawlers.” Check to see if “Test-Crawltrack” is one of the items in the crawlers list. See screen shot below:

Be sure to come back next week for the second part of this tutorial.

Google+ Comments

Google+ Comments