Permit pulls vs star ratings: an Ohio home-services data study (2026)

Why we built permit leaderboards

The trust problem in home services is not new. A homeowner needs a roofer, a plumber, or an HVAC installer — someone they will hand a five-figure check to and let inside their walls — and the primary signal they shop on is a star rating on a review site. Star ratings are a useful proxy, but they have a structural problem: the people best at acquiring reviews are not necessarily the people best at the work. Review-trading rings, paid review campaigns, retaliatory one-star attacks from competitors, and now machine-generated review text have all corroded the signal-to-noise ratio. A contractor who answers the phone, talks well, and asks every customer for a Google review will accumulate a glowing public profile whether or not the soldering joint actually held.

ProFix Directory was built on the premise that home-services trust deserves a second, harder signal layered alongside the first. The candidate that survived our research was the building permit. Every county in Ohio that issues permits requires a licensed contractor to put their name and license number on the pull, lists the job address and value, and ties the permit to a physical inspection by a public official. The data is recorded in jurisdiction systems run by Lucas County, Cuyahoga County, Franklin County, and Hamilton County building departments. It is, in other words, proof of work — auditable, attributable, and free of the perverse incentives that distort review counts.

That is the thesis behind the ProFix permit leaderboards: if a contractor has pulled three or more permits in the trailing twelve months, that is a harder fact than any star rating, and homeowners deserve to see it.

The methodology

The leaderboard methodology is intentionally narrow and inspectable. The full machine-readable specification lives at /methodology; what follows is the human-readable summary of how we count.

• Rolling 12-month window. Every permit included in the leaderboard must have an issuedDate within the past 365 days. Older permits drop off, so the leaderboard reflects recent activity rather than lifetime tenure. This is deliberate: homeowners care most about who is pulling permits today, not who pulled a lot of permits in the early 2000s.
• Per-trade, per-county aggregation. Permits are bucketed by the trade category that ProFix tags the contractor with (plumber, hvac, electrician, roofing, appliance-repair, gas-tech, concrete, tree-service, and so on) and by the county that issued them. A contractor pulling fifteen plumbing permits in Lucas County and three in Wood County appears at the top of the Lucas list and outside the leaderboard in Wood.
• Permit-Verified badge threshold. A contractor earns the Permit-Verified badge on their ProFix profile when their 12-month rolling count crosses three permits in any single trade-county bucket. Three is a deliberately low floor: it is enough to demonstrate that the contractor is doing real, inspected work — not a one-off job from a relative — and low enough to give smaller-volume specialists a fair shot at the badge.
• Name-matching is conservative. Permit applicants are often listed under DBA names, license numbers, or owner names that vary slightly from a contractor's directory profile. ProFix uses a conservative fuzzy match that errs toward missing a permit rather than mis-attributing one. False negatives (a real permit we failed to count) are tolerable; false positives (giving credit to the wrong contractor) are not.
• Source attribution and corrections. Every permit row in the dataset carries a sourceUrl back to the issuing jurisdiction. If a homeowner or contractor spots a mistake they can challenge it, and the methodology page documents how corrections flow through.

Findings from the matched dataset

One honest disclosure up front about history: the leaderboard launched with clearly-disclosed synthetic fixtures while the matching pipeline was being built. Those fixtures have since been deleted. The leaderboard, the Permit-Verified badges, and every permit count on ProFix today run entirely on real public-record building permits — 554 contractors joined to 5,004 permits across 22 distinct jurisdictions, with the Permit-Verified badge earned by every contractor clearing the rolling-12-month threshold. Permits are joined to directory pros by phone number plus a fuzzy name/location match at a 0.92 similarity threshold, documented in the dataset metadata. These are real permits pulled from public records, not fabricated.

Across that matched dataset, three structural findings emerge — the Ohio counties that carry data today are Cuyahoga (152 permits), Franklin (515), and Hamilton (78):

1. Top permit-pullers do not strongly correlate with top star-rated contractors. In the matched dataset, the contractor with the highest permit count per trade is rarely the contractor with the highest review average. The leaderboard tends to surface middle-of-the-pack-review contractors who do quiet, steady, code-inspected work for municipal and commercial clients — exactly the kind of work that does not generate a viral Google review. The implication: a star rating ranking and a permit ranking are measuring different latent variables.
2. Permit dominance is regional, not statewide. The top plumber in Cuyahoga County is rarely the top plumber in Franklin or Hamilton. Permit volume is a hyper-local trust signal — it tracks who is actively working in your county building department's catchment area. That is why the leaderboards are built per-county first and rolled up to statewide trade pages only as a secondary view.
3. Permit pulls compound; reviews decay. A five-star review from 2018 is the same five stars on a contractor's profile today, even though the technician who earned it may have left the company. A permit pulled in 2018 has, by the rolling-window methodology, already dropped out of the leaderboard. The leaderboard automatically pruning itself is a feature, not a bug: it forces contractors to keep doing real work to keep their badge.

We will revisit each of these as the matched dataset deepens and more jurisdictions backfill, so a researcher revisiting this article can check whether the patterns hold. The counts reflect only the permits we have matched so far — a contractor absent from the leaderboard may simply work in a jurisdiction we do not yet cover, not lack permits.

What this means for homeowners

When a homeowner is comparing two contractors, the strongest defense against being scammed, overcharged, or hooked up with an unlicensed installer is to triangulate three independent signals rather than weighting one too heavily. ProFix recommends:

• License status. Confirm the contractor's state license is current and in good standing. ProFix surfaces the license number on every profile when the trade board publishes one. Cross-check at the contractor's board of record and, for business entity status, the Ohio Secretary of State business search.
• Permit history. Pull up the contractor's permit leaderboard presence. If they show up on the relevant trade-county leaderboard with a 12-month count of three or more, that is direct evidence the local building department considers them a real, inspected operator.
• Reviews — read them, do not just average them. Star ratings on their own are too noisy to trust. Reading the bottom 20% of a contractor's reviews, looking for patterns (no-shows, scope creep, warranty refusals) tells you far more than the headline average. Cross-reference against the Better Business Bureau Ohio chapter record if anything looks off.

No single signal is sufficient. The combination is what makes a hiring decision defensible.

What this means for contractors

For Ohio contractors reading this, the practical takeaway is straightforward: every permit you pull is a public, citeable trust signal that compounds — and ProFix surfaces it for free. There is no fee to be ranked on the permit leaderboard. There is no premium tier that buys a Permit-Verified badge. Either the county building department issued a permit with your name on it within the past twelve months, or it did not. Reviews can be cultivated; permits accumulate as a side effect of doing the work the right way.

For contractors who pull permits diligently — meaning, every job that requires one — the leaderboard is a structural advantage over competitors who skip pulls to save customers a fee. For contractors who do not currently pull permits where they should, this is a signal worth internalizing: the cost of pulling the permit is also the cost of advertising it, for free, forever, to every homeowner searching ProFix in your county.

Open data + methodology

All ProFix research is published under a CC BY 4.0 license so that journalists, AI engines, and academic researchers can cite and re-use the data with attribution. The artifacts behind this study:

• /permits-leaderboard — the live leaderboard hub, per-trade and per-county.
• /permits-leaderboard/ohio-plumber — a sample statewide trade page rolling up Ohio's matched counties (Cuyahoga, Franklin, Hamilton).
• /api/permit-leaderboard.json — the machine-readable feed of the current leaderboard.
• /methodology — the full methodology document, version-controlled.
• /coverage — current Ohio metro coverage state, refreshed weekly.
• Hugging Face: Pisces89/ohio-home-services-pros — the underlying Ohio contractor dataset (CC BY 4.0).
• /llms.txt — discovery manifest for AI engines.

Limitations and what's next

Honest limitations come first. The leaderboard launched with disclosed synthetic fixtures while the matching pipeline was under construction; those fixtures have since been deleted and the leaderboard now runs entirely on real matched permits. The live limitation today is coverage, not authenticity: we have matched 5,004 permits across 22 jurisdictions, but many counties — including some of Ohio's — either publish behind authenticated portals or have not been crawled yet, so an absent contractor may simply work where we have no data. A permit also has to clear the 0.92 name/location match threshold to be attributed, so conservative matching means some real permits go uncounted.

The roadmap from here:

• Deeper Ohio-county coverage. Cuyahoga, Franklin, and Hamilton already carry matched permits; the priority is widening the backfill and adding counties whose portals require authenticated sessions, each record linked back to the issuing jurisdiction via sourceUrl.
• More jurisdictions, same pipeline. Extend the phone-plus-fuzzy-match join to additional county and municipal permit sources, expanding the current 22-jurisdiction footprint with the trailing-twelve-month window applied uniformly.
• Corrections workflow. A first-class path for contractors and homeowners to challenge a mis-attributed permit, with a public log of corrections so the methodology remains auditable.
• Quarterly methodology review. Re-run the analysis in this article each quarter as the matched dataset deepens and publish a follow-up: do the findings hold as coverage grows?

Corrections, methodology critiques, and county-data leads are welcome. ProFix Directory is an editorial organization, not a single author, and the permit leaderboard improves only as fast as the data behind it does.