filter non-indexable links 3
https://thezeroriskbacklinkspoint.brightpathdigest.com/posts/why-zero-risk-backlinks-are-the-missing-piece-in-your-seo-strategy
When I crawl a site, I set up a rule to filter non-indexable links—things like nofollow tags, disallowed robots entries, or pages that redirect. It saves time because I only see URLs that actually matter for indexing, and I don't waste effort on pages that won't show up in search results anyway.