Link Rot Postmortem

Ugh. So here’s what I learned while banishing broken links…

* Broken Link Checker isn’t completely intuitive, but as of this date it’s current and supported. If you have a question you may not get the answer you’re looking for, but you’ll get an answer, and you’ll profit from it.

* Over the past year the BLC plugin reported (via email) in fits and starts for reasons I did not understand. In reading up and poking around, however, I discovered a ‘server load’ setting which seems to act like a throttle. If you set it very low — meaning lower than the reported server load — you effectively idle BLC until the server load drops. Or at least I think that’s what happens. In any case, when I raised the number above the reported load, BLC sprang into action, so if you’re not getting activity when you expect it I would check that setting. (Also, if you’re on shared hosting, consider changing that setting at night when the server load is low. BLC may run much faster.)

* When you’re working on each individual broken link, going slowly and searching for missing pages on the web can be surprisingly fruitful. I fixed quite a few dead links where the missing page’s URL had been altered without a redirect. Once located, copying and pasting the current active link in place of the broken link solved the problem.

* I initially decided to deal with a minimum of twenty-five links each day, but the first day was tough. As it turned out, however, much of the struggle was due to the fact that I had no process or workflow to follow, so what I was really fighting was the learning curve, not the task. On the second day I probably fixed or killed fifty or so links, then the following day I finished off the remaining sixty or so, meaning it took me three days to get through my backlog. (If you’re in the weeds like I was, or worse, just make link-fixing a chore that you come back to again and again until it’s done.)  

* For general workflow I first opened a post which contained a broken link in a new browser window so I could see the link in context. Next, I copied the offending link and pasted it into a search box, to see if something very close might pop up. Often this only revealed other sites using the same busted link, but sometimes it took me to the same content under a slightly different URL, usually as a result of changes in the permalink.

* If searching for a link failed to turn up what I needed I next copied the base URL (e.g. ditchwalk.com) into the address bar to confirm that the site still existed. If it did, I then began running searches on that site or via a search engine to see if I could find the missing content. If a broken link contains words, those words are the first keywords you search for, because they are probably in the title as well.

* While I met with a fair amount of success overall, sometimes it was just easier to kill an entire post. I only did that a couple of times, and when I did it was not solely because of rotten links but because the posts themselves were no longer relevant. (One old post documented my frustrating search and eventual success at finding what is now commonly termed a splash screen. Another post referenced a site which is now so far removed from its original intent as to be something else entirely.) Ironically, deleting a post means anyone who linked to that content now has a dead link to deal with, but that’s the nature of the beast. Not all content is forever. (What I wouldn’t give to be able to do some heavy pruning on the internet.)

* If you find success with one URL, fix any other broken links pointing to the same URL before moving on. If you’re lucky you may be able to edit the others to match — say, because ‘/archive’ has been added to the permalinks — though you should still recheck each fix.

* If you run into a broken link that proves troublesome, skip ahead. Maybe you’ll learn something that will help when you come back to it later, or maybe you’ll just be so sick of fixing links that you’ll be ready to delete it. Either way, you save yourself a lot of stress.

* The final eight broken links were all on the same post. The site they pointed to was still up, but it had evolved so much that little of what I had written in the original post was relevant any more. Still, I wanted to keep the post precisely because of its historical value, so what to do? Axe all the links? Revise the post? Include an update? In the end I included a preface in italics to explain the reason for the missing links. While a hack, it gets the job done. (It’s also a very old post that almost no one finds anyway.)

* After dealing with all of my broken links I had another twenty warnings to deal with, which required a little different mindset, but not a whole lot of time. If you don’t like your link rot in separate categories, you can change how BLC treats warnings on the General tab for the plugin’s settings. (Uncheck the box and all warnings will be treated as broken links.)

The big takeaway, obviously, is not to get so far behind. BLC says there are thousands of links on my pages so I don’t feel too bad about the amount of rot I had to scrub away, but from now on I’ll be keeping up with the decay.

Maybe not daily or even weekly, but definitely once a month. Definitely.

— Mark Barrett