The Burp Suite User Forum was discontinued on the 1st November 2024.

Burp Suite User Forum

For support requests, go to the Support Center. To discuss with other Burp users, head to our Discord page.

SUPPORT CENTER DISCORD

Page Deduplication

Kris | Last updated: Feb 26, 2016 10:07AM UTC

Some applications offer a large set of sites that only present different data but are based on the same template. This can result in thousands of pages in the scope that are basically irrelevant. There should be some way of getting rid of similary pages or analyze the whole scope to sort out the unique pages. Gryffin from yahoo does something similar already. Something like MinHash would probably work to do the job. https://github.com/yahoo/gryffin

PortSwigger Agent | Last updated: Feb 26, 2016 01:35PM UTC