Managing SEO client files is important for both security and privacy. For example, suppose you are in the process of doing an onsite SEO work report for a certain client, and in the report, you’ve included sample web pages where you can illustrate specific onsite SEO corrective action to your client.
Of course, the most convenient way to do this is to use your own web server space to host this mockup content for your SEO client. However, there are two important, typical problems that commonly crop up for companies hosting client mockup files.
First, the mockup content is publicly accessible. This means that even Google bots and unauthorized persons can access these files — files that are supposed to be confidential.
Second, mockup pages that can be crawled and indexed by search engine bots introduce duplicate content issues with the client’s official web content. You might see this problem occur at some web development companies, where the web developer failed to blocked search engine bots, thus inadvertently allowing a client’s test website to be indexed.
As a result, after the complete deployment of the client’s website on its own web server, the site is already duplicated, with the content appearing on the web developer’s websites!
If you are a freelance SEO managing your own web server, then using Apache user authentication can be a great help in sorting out the two major issues explained above.
If Apache user authentication has been implemented on your client SEO folders(containing the necessary mockup files you need to show), then public users cannot access those files, because they are password protected. Search engine bots also will not index folders which are password protected, so you can ensure that the content will not be crawled and indexed.
You need to assign a unique username and password combination to each of your SEO clients. This way, when you provide them with an SEO report, you can include a username and password, which they can use it to access protected SEO mockup files.
So what would happen if an intruder attempted to access protected content using Apache user authentication? They would be denied access, and the server would return an “Error 401: Authorization Required” status.
Structuring Client SEO files in your Apache web server
It is important to be highly organized if you to plan to implement user authentication on your web server. It is highly recommended that you create a dedicated folder where you can place and store SEO client files.
This folder will be created at the root directory of your web server. Inside this folder is where you will create sub-folders for each of your clients. Then you can implement Apache user authentication techniques on each of these client folders so that they are password protected.
The file structure will look like this:
The above screen shot shows a web server with the following files (blue) in the root directory (index.php, robots.txt, about.php, etc).
You can see a folder named ”seoclients,” which is the dedicated folder that contains your client SEO files. Inside the seoclients folder are sub-folders for each of your clients. These green-colored folders will be protected with user authentication.
Here is what you need to do:
1. Create a folder named seoclients at the root directory of your website. Set the file permission to 755 (recurse into sub-directories) using your FTP client (Filezilla for example).
2. Create at least one sub-folder under it named myclient; set the file permission on this folder to 755 as well (also recurse into sub-directories).
3.) Open a text editor and type the following test content:
<title>Restricted Page on Myclient folder</title>
<p>This is a restricted page. If you have seen this; then you have successfully authenticated using a username and password.</p>
4. Save it as index.htm.
5. Upload index.htm to myclient folder.
.htpasswd is a file that contains the username and password. Similar to .htaccess, this file starts with a dot and does not have a file extension.
This can be created similarly to .htaccess, so a text editor can be used. For security purposes, the passwords are encrypted (using MD5 algorithm), and this will upload to a path in the server which cannot be accessed by a web browser.
In Apache web server, the root directory is the highest directory accessible by a web server. However, using SSH or FTP client, you can upload files which are outside your root directory or public html folder.
This is where you can upload your .htpasswd file. Here is an example screen shot:
The grey-colored regions are accessible by a web browser. The file .htpasswd will be uploaded NOT in the root directory, but outside it.
Here is what you need to do:
1. Go to this URL: http://www.htaccesstools.com/htpasswd-generator/
2. Try entering a test username and password.
3. Click “Create .htpasswd file.” This will automatically create the contents for .htpasswd based on the username and password provided.
4. Open a text editor (gedit in Linux or Notepad in Windows).
5. Copy and paste the contents generated by the tool to the text editor.
6. Save it as .htpasswd
Note: Bear in mind that there is dot “.” character before the filename, and there is no file extension.
This is how the contents of the .htpasswd will look:
7. Using SSH or FTP client, upload it outside the root directory. If you have problems uploading files above the root directory or issues with .htpasswd, you might need to contact your web hosting support for guidance, or the try the “alternative method” I explain below.
Alternative method: Some web hosting companies do not allow uploading of files outside the root directory, or .htpasswd will not work even though it is correctly uploaded outside the public html. In this case, you will need to upload .htpasswd to the root directory /public html inside of a protected folder.
- Create a folder named accessdetails in your website root directory, and use a file permission of 755.
- Place .htpasswd inside accessdetails folder, set the file permission of .htpasswd to 644.
- Upload an .htaccess inside accessdetails folder that contains the syntax below:
deny from all
The file permission of the .htaccess can be set to 644. Please refer to this SEO Chat tutorial for the creation of .htaccess.
So inside the accessdetails folder are two files, .htpasswd and .htaccess. The purpose of .htaccess is to protect the .htpasswd from public access, since it is uploaded NOT outside the public HTML folder.
Create .htaccess to authenticate access on your SEO Client folders
Finally, you need to create a .htaccess, which you will place inside your SEO client directory (“myclient” folder in this example).
The syntax of the .htaccess will be:
AuthName "SEO Project Mockups"
allow from all
Replace the path to the .htpasswd with your own path. In the example above, the path to the .htpasswd is: /home/www/php-developer.org/accessdetails/.htpasswd
You can also assign as different Authname, so in this case change the text to ”SEO Project Mockups.”
So your client folder (myclient for example) will contain both SEO client files (e.g index.htm), and .htaccess containing the syntax above (path to the .htpasswd).
So how are you going to add additional user authentication for new SEO client folders? You need to create a separate .htpasswd for each client to give them unique access.
For myclient : /home/www/php-developer.org/accessdetails/.htpasswd
For another client: /home/www/php-developer.org/accessdetails/.htpasswd 1
Where .htpasswd1 is the .htpasswd file (containing a different username and password) for another client folder.
Of course the .htaccess inside the “another client” folder will be:
AuthName "SEO Project Mockups"
AuthUserFile /home/www/php-developer.org/accessdetails/.htpasswd 1
allow from all
The AuthUserFile should point to its specific and correct .htpasswd file.