Semrush is a full Website positioning suite that gives an awesome website audit tool. Conveniently named Site Audit, it’s an alternative choice for crawling your site. As the title suggests, it is actually an extensive auditing tool, not merely a crawler. I have written custom made crawling and Investigation https://youtu.be/EVeTP0aV8gY?si=5FHbTy4kbLwUVoJD