Turning Audit Findings into CI Checks
You get a site audit report and it looks manageable. A few dozen findings, most of them small: a page with barely any text on it, a link whose text is just “here”, a page whose title tag is a copy of its H1, a hero image heavy enough to hurt the largest contentful paint . None of it is that hard. You spend an afternoon on it, close the tickets, and move on.
Then a few months pass, a dozen new pages ship, and the next audit reports the same findings again. Not because anyone ignored the first round, but because the first round fixed pages instead of fixing the process that produces pages.
That happened to us, on this site and on OmbuLabs.ai . So the second time around we spent the effort somewhere else. Instead of just fixing the pages, we wrote checks that run on every build and say when a new page has the same problem. It is roughly the same idea as automating a tech debt audit : most of the value is not in the report, it is in being able to produce the report again for free. No two of them wanted the same kind of check.
The goal here is search traffic, not a clean report. Thin pages, vague link text, and duplicated title tags are the things that hold a page back in search results, and a page that ships with them costs us traffic until the next audit finds it. A check on every build moves that discovery from months later to the pull request.
In this article, you will learn how we turned three kinds of audit findings into checks that run in CI.
Word count, text ratio, and semantic ratio
The first group of findings was about the content itself, the pages a crawler treats as thin. By crawler I mean Googlebot, Bing and the AI crawlers that feed answer engines. There are three metrics that matter here:
Word count. How many words the page actually says.
Text ratio. The size of the visible text divided by the size of the whole HTML file. A page can look full to you and still be 5% words and 95% tags, and a crawler reads that as a page built to hold a layout rather than to say something.
Semantic ratio. How many of the page’s tags are ones that carry meaning (h2, p, ul, article, footer) instead of plain div and span. Semantic tags are what tell a crawler which part is a heading and which part is the body. A page made of nested divs says nothing about its own structure.
We put all three in one class, ContentQualityChecker, that parses the page with Nokogiri, with the thresholds at the top:
class ContentQualityChecker
WORD_COUNT_MIN = 200
TEXT_RATIO_MIN = 0.10
SEMANTIC_RATIO_MIN = 0.115
SEMANTIC_ELEMENTS = %w[
header nav main article section aside footer figure figcaption time mark details summary
h1 h2 h3 h4 h5 h6 ul ol li p blockquote table tr td th
].freeze
Those numbers are not set in stone. They are in the range the common audit tools complain about, and they are low enough that a real article never trips them, which is what you actually want from a threshold. If yours flag half your posts, they are set too high for your layout.
The one number that needs care is the word count. Counting the whole <body> sounds right but it is not, because every page carries the same nav, sidebar, and footer, and on our pages that added roughly 40% to the number. Thin pages looked fine. So the count cuts out everything but the main content area, doc.at("main"), falling back to the body for anything without one.
With the checker in place the rest is two rake tasks, and they do not work the same way. Our blog is a Jekyll build, so the pages are already HTML on disk and that task just globs the files and needs nothing but Nokogiri. Rails pages only exist once something renders them, so that task opens an ActionDispatch::Integration::Session and requests each path the way a visitor would. It is the cheapest way we found to measure a rendered page without a browser in the loop.
The report prints one line per flagged page, worst first:
/blog/tags/railsconf 195w text 11.0% semantic 16.2% low word count
/blog/fortify-rails-security-webinar 280w text 9.7% semantic 25.8% low text ratio
Today it flags 20 pages out of the 304 it measures on the blog, and most of them are edge cases we are fine with. Tag and author listings are thin because they are navigation, not articles. A group of long posts from years ago trips the semantic ratio because their markup is older than the current layout. Expect a tail like that of your own, every site has one. The report is there to tell the pages that are thin on purpose from the one somebody shipped last week without noticing.
Anchor text that describes the destination
The second group is the interesting one, because the audit finding and an accessibility bug turn out to be the same bug.
Anchor text is the visible words inside a link. A crawler just extracts that text, so click here gets read as exactly that, “click here”, with no information about where the link takes you. A screen reader user gets the same nothing, out of context, from a list of links on the page.
Two findings came up. Links with no anchor text at all, and links whose anchor text describes nothing: here, click here, read more, learn more, details, download, continue. If you have read anything about accessibility, and I covered some of this in Usability Meets Accessibility and From Code to Compliance: Accessibility Testing , that list is familiar. The way people usually try to fix the first one misses the point.
Take an icon-only link, a social icon in the footer for example. It has no text node at all. The first thing people try is an aria-label on the <a>, or alt text on the image inside it, and then they consider it handled. For a screen reader that mostly works. For a crawler it does not, because it reads the anchor’s text content, and neither an attribute on the link nor an attribute on a child image is text content. The fix that works for both readers is a real text node that happens to be invisible: a <span> with the sr-only class, a CSS convention that pushes text off screen while leaving it in the document. The name says “screen reader only”, but it fixes crawlers too: both read text content, not what is visible on screen.
Our scroll-to-top link is the smallest example. Before, an image and an attribute:
<a class="scrollto" href="#top">
<img src="circle-black-lg.svg" alt="Scroll to top">
</a>
After, the image is marked as decoration and the words live in the document:
<a class="scrollto" href="#top">
<img src="circle-black-lg.svg" alt="" aria-hidden="true">
<span class="sr-only">Scroll to top</span>
</a>
Nothing changes on screen. The difference is that the second link has anchor text and the first one has none.
Then there was the part that gave me the most headaches, and it came from one of our own gems. Our blog builds with jekyll-external-link-accessibility , a plugin we maintain that appends a note to every external link so screen reader users know it opens a new tab. That is a good thing to do, and it also means that by the time the checker sees the HTML, a link reading “here” reads “here opens a new window”, which matches nothing. Every non-descriptive link on the blog passed clean. So the text has to be cleaned up before it is judged:
DECORATION = /\s*opens a new window\s*\z/i
def visible_text(anchor)
stripped = anchor.dup
stripped.css("[aria-hidden='true']").each(&:remove)
stripped.text.gsub(/\s+/, " ").strip.sub(DECORATION, "")
end
The aria-hidden removal is the same idea running the other way. An icon marked aria-hidden="true" is decoration nobody hears, so it should not count as anchor text either.
A baseline spec for page metadata
The third group was metadata. A rake task is the wrong tool here. There is no threshold to report on, the tags are either present or they are not. So this one is a request spec. It requests each public page and asserts the tags a crawler and a link preview need:
aggregate_failures("SEO/social baseline for #{path}") do
expect(body).to match(/<meta\s+name="description"\s+content="[^"]+"/),
%(Missing or empty <meta name="description"> on #{path}. Add static.<action>.description in config/locales/en.yml.)
expect(body).not_to match(/translation missing/i),
%(Found an unresolved I18n key ("translation missing") on #{path}. Add the missing key in config/locales/en.yml.)
expect(body).to match(/<meta\s+property="og:url"\s+content="http[^"]+"/),
%(Missing or non-canonical og:url on #{path}. It should be the full URL of the page, not a hardcoded value.)
end
There are a dozen more like that, covering the rest of the og: tags and the JSON-LD blocks. aggregate_failures reports everything wrong with the page in one run, and every message names where the fix goes, down to the locale key. A spec that fails with “expected false to equal true” gets skipped by the next person who hits it.
None of that matters if the spec never visits the page. Keep a list of the paths left out on purpose, with a reason for each one, so a new page never slips through unnoticed:
IGNORED_PATHS = {
"/search" => "redirects to /blog when query is blank, so does not render HTML",
"/success" => "redirects to /roadmap after Stripe checkout",
"/robots.txt" => "non-HTML",
"/sitemap.xml" => "non-HTML"
}.freeze
Writing the reason down is worth more than it looks. An ignore list without reasons becomes a place to hide failures, and six months later nobody remembers whether /success is skipped because it redirects or because someone was in a hurry.
The list only works if it cannot go stale, so one example asks the router what exists and compares. Trimmed a little:
it "covers every public GET route or lists it in IGNORED_PATHS" do
discovered = Rails.application.routes.routes.flat_map do |route|
next [] unless Array(route.verb).first.to_s.include?("GET")
spec = route.path.spec.to_s.sub(/\(\.:format\)\z/, "")
next [] if spec.match?(/[:*]/)
controller = route.defaults[:controller].to_s
next [] if ignored_controller_prefixes.any? { |prefix| controller.start_with?(prefix) }
[spec]
end.uniq
expect(discovered - (PUBLIC_PAGE_PATHS + IGNORED_PATHS.keys)).to be_empty
end
Routes with parameters are skipped, since you cannot request them without making up a record, and so are admin, API, and framework controllers, since nothing there is meant to be indexed. Everything else has to be in one list or the other. Add a public page and the spec fails with the path in the message. It fails the other way too, when a listed path no longer exists in routes.rb, so old pages cannot stay in the list.
Conclusion
In this article, we went through three ways to keep an audit finding from coming back, so the pages we ship keep earning search traffic instead of losing it a little at a time. By setting a concrete metric or rule for each kind of finding, a word count and markup ratio for content, a matching tag for metadata, we could check it automatically in CI on every pull request. That let us tell which problems needed fixing right away and which ones were fine to track and fix over time.
Automate the metadata spec first. It pays for itself the first time somebody adds a page in a hurry.
Is your team carrying a backlog of findings that nobody has time to keep fixed? We can take that off your hands , send us a message.