Below is a personal list of web spiders and web crawlers that I block from crawling various sites.
This helps mitigate issues related to server load, security, and content scraping.
Carefully consider the impact on your website’s visibility and search engine ranking before applying any of my crawler blocks within this article.
By default, WordPress saves every change you make to a post or page as a revision.
While this is great for version control, it can quickly bloat your database—especially on long-form content or frequently edited pages.
In fact too many revisions can slow down your admin dashboard, cause memory issues with certain page builders (like Avada or Elementor) as well as lead to database bloat over time.
This article is a quick reference for how I automatically limited the number of stored WordPress revisions.
In today’s digital landscape, securing your website is more than just a best practice—it’s essential. One powerful tool in your web security arsenal is HTTP Strict Transport Security (HSTS).
This policy ensures that browsers only connect to your site using secure HTTPS connections, protecting users from threats like protocol downgrade attacks and cookie hijacking.
This tutorial walks you through implementing HSTS on your website using the .htaccess file and submitting your domain to the HSTS preload list for maximum protection.
Here are three techniques to increase your website’s security posture.
This tutorial explains how to add X-Security Headers to protect against cross-site scripting (XSS), page-framing, and content-sniffing by adding extra security headers to your sites’ resources.
Hotlinking is the act of linking to a file that is hosted on another site, instead of downloading the file and hosting it on your own server. With proper source citation of course.
Images are most frequently hotlinked filetype, but zip, PDF and other digital assets can also be hotlinked.
Since hotlinking requires the website server hosting the file to use its own bandwidth to load it on the offending site, it’s considered bad etiquette. Others go further and call it theft. That’s because it eats up the bandwidth of the site you took it from without giving them the benefit of increased traffic or credit.
In this article I show you how to block others from hotlinking various filetypes from your web server. To spare your server from other undesired traffic, read my article on a list of web crawlers to block.
The Facebookexternalhit/1.1 is a user agent used by Facebook to crawl and index web pages for its various services, such as Facebook, Instagram, and WhatsApp.
This crawler is responsible for retrieving content, images, and other metadata to improve Facebook’s search functionality and provide users with relevant results.
However, the facebookexternalhit crawling activity can sometimes cause issues for website owners, such as increased server load, bandwidth consumption, and potential security risks.
This article covers why you might want to block facebookexternalhit as well as how to block the facebookexternalhit crawler.
Recently was not able to convince a client of the benefits of having me professionally manage, host and maintain their website for them. They wanted to save money and use GoDaddy’s Basic Managed WordPress service. So I had to migrate their completed demo over to it but ran into countless errors with GoDaddy’s hosting platform.
The GoDaddy migration tool was useless as was GoDaddy support. GoDaddy support insisted that I upgrade the server in order to complete the migration. But I only had delegated access to the clients account with no ability to purchase said upgrade because the client did not grant me those permissions.
Long story short, I was able to manually migrate their site over, even though we were on the Basic Managed WordPress server using the steps enumerated in this article.
This article is only for my personal use, to help jog my memory if another client want to migrate their site to a more affordable solution. Please do not use this article for migrating any of your or client sites. If you do, you take full responsibility for anything that goes wrong, or right for that matter.
Recently needed to implement a local development environment that used Apache, PHP and MySQL on Windows.
Years ago I played with WAMP server, which worked well. But the payload was large (1GB installer), system resource utilization high, and it required a number of outdated Microsoft Visual C++ versions to be installed in order to operate correctly. It was also very slow to use but it did work.
Local by Flywheel was slightly better but also very, very slow on a Windows computer. At least in my experience.
By far the best experience has been XAMPP. Granted, you have to run the installer as an administrator as well as its primary executables. But beyond that, it had a smaller installer and opening up a local WordPress installation is at least 4x faster than with Local by Flywheel. In short, for my personal needs the clear winner is XAMPP. Your experience may be different.
Within this article I have instructions and a video covering how I installed XAMPP on my Windows PC for quickly spinning up a WordPress installation for use in WordPress Plugin development.
There may be some situations when you want to restrict or redirect your website traffic by IP address. To accomplish this simply add the following lines of code to your website’s .htaccess file (assuming your website is on an Apache webserver):
We were recently contracted by a new client to “fix their website” which suddenly went offline.
Our analysis revealed that they had an improperly configured Apache .htaccess file. This can happen if the hosting company migrates a website or updates the .htaccess file, which they do from time to time, but without testing for compatibility. This can also happen for a number of other reasons but you aren’t here for those.
The client authorized us access to their BlueHost hosting account so that we could resolve the issue for them. It was at that time that we discovered that their .htaccess file was not visible from within the File Manager component window under their cPanel.
Usually one would just check the box to “show hidden files” under the options menu which would reveal the .htaccess file. In this case there was no such option.
It was then that we discovered a work around which would force the reveal of any hidden files from within the File Manager component window. We will share this tip with you below.
