Menu

Technical SEO

Technical SEO Audit Workflow

A technical SEO audit workflow for validating crawl access, indexability, canonical URLs, rendering and site signals with reproducible evidence.

Published
Updated
Reading time
7 min

Difficulty

intermediate

Works with

Screaming Frog, Sitebulb, Google Search Console, browser devtools

Inputs

  • Crawl exports and configuration
  • XML sitemaps
  • Search Console evidence
  • representative templates and server responses

Outputs

  • Issue inventory
  • reproducible evidence
  • prioritized remediation plan
  • release validation checklist

What this workflow does

This workflow turns crawler exports, server responses, rendered pages and Google Search Console evidence into a technical issue model. It deliberately separates discovery, crawl access, successful fetching, indexability, canonical selection and ranking because those are different stages with different evidence.

Google’s minimum technical requirements are narrow: Googlebot must not be blocked, the page must return a successful response, and it must contain indexable content. Meeting those requirements makes a page eligible, but does not guarantee crawling, indexing or visibility.

When to use it

Use it for:

  • a site-wide baseline before a roadmap is agreed;
  • a redesign, domain change, platform migration or URL migration;
  • an unexpected indexing or canonicalization problem;
  • a new template, JavaScript rendering change or faceted navigation rollout;
  • release validation after technical SEO fixes.

For a single symptom, narrow the scope. A smaller investigation with stronger evidence is more useful than a site-wide crawl followed by generic warnings.

Define scope before crawling

Document the properties of the audit before collecting data:

Scope field Record
Hosts and protocols Production hosts, subdomains, HTTP/HTTPS variants and known aliases
URL policy Trailing slash, lowercase, parameters, international variants and preferred canonicals
Crawl settings User agent, robots handling, JavaScript rendering, speed, limits and authentication
Search scope Countries, languages, device assumptions and Search Console properties
Priority templates Home, category, product, article, location, utility and conversion pages
Exclusions Staging, account, cart, internal search and intentionally blocked areas
Evidence date Crawl time, exports used and release version

Create a stratified sample across templates and states. Include successful pages, redirects, not-found URLs, non-indexable pages, parameter variants and JavaScript-dependent pages where relevant.

Audit sequence

Confirm that important pages are reachable through normal HTML anchors with real href values. Compare the rendered internal-link graph with the canonical URL inventory and XML sitemap.

Flag:

  • important pages with no crawlable internal path;
  • navigation that appears only after unsupported interaction;
  • broken destinations, redirect chains and links to non-canonical variants;
  • important pages buried behind avoidable depth or faceted combinations;
  • mobile navigation that exposes fewer essential links than desktop.

A sitemap helps discovery, but it does not replace a coherent internal-link structure.

2. Crawl access and directives

Fetch robots.txt directly and test representative allowed and disallowed URLs. Record the user agent used for the decision. Separate crawl controls from index controls: Google documents robots.txt primarily as a crawl-management mechanism, not a reliable method for removing a web page from Search.

Check page-level robots meta tags and X-Robots-Tag headers on HTML and non-HTML resources. Look for conflicts between templates, headers and rendered HTML. If a page needs noindex, Googlebot must be able to crawl it to see that directive.

3. HTTP responses and redirects

Capture the full response chain for each sample, including status, location, final URL and loops. Verify that:

  • indexable pages return 200 with the intended content;
  • removed pages return a meaningful 404 or 410 rather than a soft error page;
  • permanent moves use a direct permanent redirect to the closest relevant destination;
  • temporary states are not accidentally made permanent;
  • redirect targets are indexable, canonical and internally linked.

Do not report every redirect as a defect. Judge whether it is intentional, direct and consistent with the preferred URL policy.

4. Indexability

For each representative URL, record all factors that could prevent indexing: response status, robots directives, authentication, unsupported content type, empty or error-like content and policy concerns.

Use Search Console’s URL Inspection and Page Indexing evidence for Google-specific conclusions. A third-party crawler can show what it received; it cannot prove which URL Google indexed or selected as canonical.

5. Canonicalization and duplicates

Compare the declared canonical, final response URL, internal links, sitemap entry, hreflang references and Google-selected canonical where available. Google treats redirects and rel="canonical" as strong canonical signals and sitemap inclusion as a weaker one. When these signals agree, they give Google a clearer preference, but canonical selection remains Google’s decision.

Review duplicate groups by template and cause:

  • protocol, host, case or trailing-slash variants;
  • tracking and sorting parameters;
  • print, filter and pagination variants;
  • near-identical regional pages;
  • duplicate pages created by platform routes.

Do not canonicalize genuinely distinct pages only because they target related queries. The canonical should represent duplicate or very similar content.

6. XML sitemaps

Validate syntax, response codes, encoding and sitemap index relationships. Confirm that submitted sitemaps contain absolute, preferred canonical URLs that are eligible for indexing. Remove redirects, error URLs, duplicates and intentional noindex pages.

Check lastmod only if it reflects a meaningful content update. Do not update dates merely to imply freshness, and do not interpret sitemap inclusion as an indexing guarantee.

7. Rendering and primary content

Compare the raw response, rendered DOM and visible page. Verify that titles, meta descriptions, canonical tags, robots directives, primary copy, headings and important links survive rendering and do not conflict.

For JavaScript sites, inspect blocked scripts and APIs, soft error states, client-side redirects and content that requires user interaction. Google can render JavaScript, but server rendering or pre-rendering can reduce dependencies for users and crawlers. Diagnose the actual failure before prescribing a framework change.

8. Titles, snippets and headings

Sample each template for a unique, descriptive <title>, one clear main heading and a useful meta description. Google may generate title links and snippets from several page signals, so treat these elements as strong inputs rather than fixed SERP copy.

Flag boilerplate, empty values, excessive repetition and titles that do not match the primary language or visible topic. Do not enforce an arbitrary character count; evaluate clarity and distinctness on representative devices and query contexts.

9. Structured data

Validate JSON-LD syntax and compare every material property with visible page content. Use a Google-supported feature only when the page type and content meet its specific requirements. More markup is not inherently better, and valid schema.org vocabulary does not guarantee a Google rich result.

Record both technical validity and content eligibility. Test with the Rich Results Test and confirm discovery through Search Console after deployment.

10. Page experience and mobile checks

Test essential templates on mobile widths and real devices where possible. Review HTTPS, intrusive interruptions, layout stability, loading and interaction. Core Web Vitals are part of the wider page experience; they should not replace relevance, accessibility or useful content.

Use field data for user-experience conclusions when sufficient data exists. Lab data is diagnostic and may not represent real users.

Prioritize findings

Use a consistent decision model instead of ordering issues by crawler severity labels.

Factor Question
Importance Does the issue affect pages that serve a valuable user or business task?
Reach How many important pages and templates are affected?
Evidence Is the issue reproduced in responses, rendered pages or Google data?
Expected effect What discovery, indexing, presentation or user problem should the fix resolve?
Effort and risk What dependencies, regressions and rollout controls are involved?
Validation How will the team know the intended state shipped and was processed?

Classify urgent blocks separately from improvements. An accidental site-wide noindex is not comparable to a missing meta description on a low-priority page.

Finding template

Every finding should contain:

  1. a concise problem statement;
  2. the affected rule, template and sample URLs;
  3. request, response, rendered or Search Console evidence;
  4. the expected effect on people or search processing;
  5. the likely root cause and confidence level;
  6. the desired end state rather than a repeat of the tool warning;
  7. an owner, rollout note and post-release validation.

Quality checks

  • The crawl configuration and evidence date are included.
  • Google-specific claims use Search Console or direct Google documentation where appropriate.
  • Crawlability, indexability, canonicalization and ranking are not used as synonyms.
  • Every finding includes reproducible evidence and representative URLs.
  • Robots, canonical, redirects, sitemap and internal links are checked for consistency.
  • Structured data matches visible content and a relevant supported feature.
  • Priority reflects important affected pages and confidence, not URL count alone.
  • Every remediation includes a release and post-processing validation step.

Limitations

A crawl is a time-bound sample. It may miss logged-in content, intermittent failures, Googlebot-specific behavior, URLs with no discovery path and changes deployed after collection. Search Console reports can also aggregate or lag behind live changes.

State missing access, incomplete samples and contradictory evidence. Do not turn uncertainty into a definitive diagnosis.

Google documentation used

Pair this workflow with the Technical SEO Audit Playbook for sequencing, ownership and release management.

Prompt and skill content

Objective: Audit the supplied website for technical SEO risks using reproducible evidence.

Inputs:
- Crawl exports with crawler settings, user agent, start URLs and crawl time
- XML sitemap URLs and robots.txt
- Search Console indexing, URL Inspection and performance evidence
- Representative raw and rendered HTML samples
- Preferred host, protocol, URL and canonical rules

Method:
1. Confirm scope, business-critical templates and the preferred URL format.
2. Validate a stratified URL sample manually before extrapolating crawl findings.
3. Separate discovery, crawl access, HTTP response, indexability, canonicalization, rendering and serving findings.
4. Compare signals across internal links, redirects, canonicals, sitemaps and hreflang where applicable.
5. Group issues by root cause and template; attach an example request, response and rendered check to every finding.
6. Prioritize by affected important pages, evidence strength, likely user or search impact, implementation effort and regression risk.
7. Define an owner, desired state and post-release validation for every recommendation.

Guardrails:
- Do not infer indexation from a crawl alone or impact from status codes alone.
- Do not use robots.txt as an index-removal recommendation.
- Treat rel=canonical and sitemap inclusion as signals, not guarantees.
- Do not recommend structured data that is unsupported, misleading or absent from visible page content.
- Flag missing access or contradictory evidence instead of filling gaps with assumptions.

Return an executive summary, scope and methods, findings table, prioritized remediation plan, validation plan and limitations.

Site search

Find research and resources

Type at least two characters to search.