Gillish / Mesh docs

Planned capability map

What Gillish Mesh will do.

Gillish Mesh is a parked Phase 0 skeleton. Nothing on this page is shipped. This is a planned-capability map distilled from the implementation plan and the product brief, written so a future reader knows exactly what the tool is meant to become, not a how-to for software that exists. Every section below describes intended behaviour in the future tense; when the phases ship, this page is rewritten into real user documentation in the same chrome.

Read this first#

Mesh will be a single-purpose WordPress plugin that crawls a competing blog, extracts its internal-link graph, computes structural metrics, and renders a side-by-side comparison against your own site. It is the standalone sibling of Gillish Node and shares no code with it, the only contact surface is an optional one-way read of Node's graph cache. It is aimed at niche bloggers (food, fitness, tech reviews) on 100–2,000-page WordPress sites who can't justify Ahrefs / Semrush / Screaming Frog pricing. It is deliberately not a real-time crawler, not a Screaming Frog replacement, not a multi-competitor overlay in v1, and not a topical-NLP analyser.

What Mesh will be#

The plugin will add a single top-level admin menu entry with three sub-pages:

  • Dashboard: a table of past crawls plus a new-crawl form. First crawl should be startable within 30 seconds of activation; there will be no mandatory configuration wizard.
  • Crawl Detail (per crawl), three tabs: Their site, Your site, and Compare. Tab state lives in the URL so it survives reloads and can be deep-linked.
  • Settings: a single page, no tabs in v1.

Crawls run in the background (5–25 minutes), so progress, ETA, and current state will be visible at all times, silent waiting is the single biggest support driver for tools like this.

The crawl pipeline#

A crawl runs in the background in small batches (25 pages at a time), picking up where it left off until the whole site is done:

  • Find the pages: Mesh reads the site's sitemap to get the list of pages. If there's no sitemap or it's broken, it tells you clearly (in v1 you paste the sitemap address yourself; reading a site without one is a later idea).
  • Fetch the pages: Mesh visits each page the way a normal browser would. It pauses if Cloudflare blocks it, and stops if the site returns too many errors in a row.
  • Read the links and build the map: Mesh reads the links inside the main content of each page, not the menu or footer, so navigation links don't drown out the real ones. It tidies up the web addresses so duplicates collapse, then works out the link map and the page levels.

Good manners are locked on: one page per second per site (with a little randomness), robots.txt always fetched and respected (no opt-out in v1), a 1,000-page default limit (hard ceiling 5,000), files that aren't web pages skipped, and the plugin's address sent with every request so a site owner can see who is crawling them in their logs.

Cloudflare handling#

Many target blogs sit behind free-tier Cloudflare. The strategy will be three tiers, honest about its limits:

  • Look like a real browser by default: Mesh identifies itself as Chrome on Windows, which gets past Cloudflare's basic check on roughly 70–80% of free-tier sites.
  • Per-crawl custom-headers / cookies field: paste your own browser session cookies from DevTools when a target is harder; manual but reliable for free-tier Cloudflare. These cookies are stored encrypted and only unlocked the moment they're needed, never written to any log.
  • Detect-and-explain: when a block page is detected (an error code or a known Cloudflare block page), the crawl pauses and the notice explains how to paste cookies, it says "Cloudflare is blocking us, here's how to get past it", never just "403 Forbidden".

Hard challenges (JS challenge, Turnstile, paid WAF, "I'm Under Attack Mode") are not solvable from PHP. The plugin will degrade gracefully and say so, it will not pretend it can defeat real protection.

Their site / Your site / Compare#

Their site will show the competitor's structure as an interactive map, a box for every page, a line for every link, with hover labels, zoom and pan, click to focus, and colour by level.

Your site will show the same view for your own site. Without Gillish Node, you point Mesh at your own sitemap once during setup; it runs the same crawler against your site and caches the result as a special "self" crawl (so the graph is apples-to-apples, same DOM scope, same anchor extraction). With Gillish Node installed, it auto-loads from Node's graph cache instead (see Better Together).

Compare will be the headline view, the default landing tab when a crawl finishes, because the comparison is the value you bought the plugin for. "Their site" is secondary.

Comparison metrics#

The Compare tab will render a metrics table, your site vs. the competitor, with a signal column, covering: total pages in graph, average internal links per page, Level 1 hubs, Level 2 pillars, average hub Link Gravity, orphan-page ratio, longest internal chain, cross-cluster link density, and the top anchor text on hubs. Each ⚠️ row will be clickable and jump to a pre-filtered view on the relevant tab, for example "competitor has 8.7% cross-cluster density vs your 2.1%" → click → see the actual cross-cluster edges. The vocabulary mirrors Gillish Node so the two read as siblings: Link Gravity, Levels, Hub, Cluster, Orphan, Cross-cluster density, plus one Mesh-only term, Structural delta.

Better Together with Gillish Node#

Mesh will work end-to-end with Node uninstalled. When Node is installed (and at least v2.78.0), Mesh quietly notices and adds extra features at no cost to people who don't have Node:

  • Your site loads straight from Node: instead of a wasteful ~15-minute crawl of your own site, Mesh reuses the map Node already built, so you compare against the exact same map you see in Node's Content Graph.
  • Cross-navigation buttons: jump between Mesh's "Your site" tab and Node's Content Graph.
  • Shared dark-mode default: if Node is active and you've set Node to dark, Mesh defaults to dark on first load (still overridable per plugin; no two-way sync).

What will not happen: Mesh never writes to Node's data, never changes how Node behaves, and never needs Node to work. The link is one-way and read-only. Every shared feature is a bonus, never a requirement.

Privacy & GDPR#

The plugin developer is not the data controller. The WordPress site owner who installs Mesh and starts a crawl is. The plugin's job is to make GDPR-compliant use possible: pasted cookies stored encrypted and never logged; crawl data plugged into WordPress's built-in privacy tools, so it shows up in the standard Tools → Export / Erase Personal Data flows; an automatic cleanup that deletes old crawl data after a set time (default 90 days); a clean uninstall that removes everything Mesh stored, leaving nothing behind. A first-use disclaimer will fire once per WP user before the first crawl can start, it explicitly names the controller role, the robots.txt enforcement, and how pasted cookies are protected and wiped. The in-plugin docs will include a copy-paste privacy-policy paragraph the site owner can use.

Settings#

A single Settings page (no tabs in v1) will expose: max pages per crawl (default 1,000, hard cap 5,000), request interval (default 1,000 ms), the User-Agent string, Accept-Language, retention days (default 90), the robots.txt respect toggle (locked on in v1, with a tooltip), the URL-normalisation parameter blocklist, and the per-user colour scheme (auto = follow OS, or follow Gillish Node's setting when detected).

For the build sequence behind all of the above, see the Foundation plan; for the strategy behind the product, the Product brief. The in-repo plans/GILLISH-MESH-PLAN.md is the canonical technical design and the ultimate source of truth.

↑