Making a site retrievable

Technical SEO is not an optimisation. It is a set of conditions that must hold before anything else you do can matter. It is the floor, not the ceiling — and a clean floor wins nothing on its own.

The conditions

The page returns HTTP 200. Google indexes successful responses. Error pages are not indexed. Soft 404s — a "not found" message returned with a 200 status — are worse than a real 404 because they waste crawl and can get indexed.

Crawling is allowed. robots.txt controls fetching. It is a crawl directive, not a privacy or index control: a blocked URL can still appear in results if enough links point at it, because Google knows it exists without being able to read it.

Indexing is allowed. noindex in a meta tag or HTTP header keeps a page out of the index. Remember the trap from module one: if the URL is blocked in robots.txt, Google cannot fetch the page to see the noindex.

The canonical is right. When several URLs serve near-identical content — tracking parameters, pagination, HTTP and HTTPS, trailing slashes — rel="canonical" tells Google which one to consolidate on. Getting this wrong scatters signals across duplicates or, in the worst case, canonicalises your money page to something trivial.

Content is in the rendered HTML the mobile crawler sees. Client-side rendered content goes into a render queue and lands later. It usually works; it is also the most common cause of "Google sees a blank page".

The site is discoverable. A sitemap lists what you want crawled and helps on large or poorly-linked sites. It does not compel indexing. Internal links matter more: a page with one or two internal links pointing at it is telling Google it is unimportant, and "Discovered – currently not indexed" is often a link-depth problem rather than a content problem.

Structured data

Schema markup does not improve ranking. It makes a page eligible for rich result appearances — review stars, FAQs, product prices. Eligible, not entitled. Treat it as a presentation feature with a real CTR payoff, not a ranking lever.

Speed and Core Web Vitals

Real. Modest. A genuinely slow site loses users and gets crawled less; a fast site does not out-rank a better answer. Fix speed because it costs you conversions, and take any ranking benefit as a bonus.

What trips people up

• robots.txt to remove a page from results. Wrong tool. Use noindex.
• Auditing endlessly. A perfect technical audit on a site with nothing worth ranking produces nothing.
• Ignoring internal links while obsessing over external ones.

You have got this when

Given a page that will not index, you can work through status code, robots, noindex, canonical, rendering and internal links in that order without guessing.