Research Academy
My account Customers About Blog Careers Legal Contact Resellers

Burp Suite User Forum

Create new post

Follow HTTP stream in proxy history and better relationship visualisation

Sylvain | Last updated: Jul 08, 2021 07:22AM UTC

Hi, Ok, let me start by saying I don't even know how to describe what I am requesting but in a nutshell It would be something similar to the "follow TCP stream" in Wireshark or the Maltego relationship graph... Confused? makes no sense? I will try to describe what I am asking here (which is probably impossible but hey!) The goal: To make it easier to hunt for issues when you look back at your proxy history in BURP suite The context: When I do a pen test on a web site and if I have some test user credentials, I usually start by login in the web portal, and "lightly" intercepting the requests made... just to get a peek and a feel on the website is secured (or not). I usually don't modify anything... Then I do a few more passes and I may start fiddling with some of the HTTP requests' parameters, use some of the BURP extension, etc... Then there is a 3rd pass and I look at what BURP is warning me about... This means more tests trying to exploit those issues. All of this creates a lot of proxy history requests. The problem: Sometimes I know I have modified a parameter in a past intercepted request and although it didn't quite succeeded, the results were interesting and I want to go further, or another vulnerability pop up and now I want to combine them, etc.. I can start my test all other again (which is what I usually do) but sometimes I also just go back in my proxy history, sort by modified requests and find the request I am interested in. But then it is difficult to really see the "thread" from that request to follow what happened before/after I did that parameter change. Sure, I can look at the request number, then go back to a "req #" sort and see the request before and after. But then there might be other requests in between that have nothing to do with that one request/thread I am trying to inspect. For exemple from requests to sub domain or external beacons, etc. Basically I found the history very useful but also very difficult to navigate after a large amount of time (and requests) and on busy/noisy sites. The potential solution: So maybe I am not using the right method... or there is already a filter/visualisation method out there I am unaware of. But I would love to get more of that information trove which is in the proxy history (especially if you don't have anymore access to your target). Rather than having just a sequential list of events/requests, I would like to see something more "clever". What? how? I am not sure :) But it would be to have at least an option to get a different visualisation! Maybe some kind of hierarchy? with nested requests? sorted by "relationships" i.e.: http://mainsite.com -> (click on it) -----> Facebook beacon -----> mainsite.com/second request -----> ccs -----> mainsite.com/3rd request ....... What I am talking about is different from the Target list where everything is listed as per the "filesystem" hierarchy I would like to see something similar to the target list format but with the proxy history Maybe that's impossible... and maybe it would be just a mess when you get similar requests to the same resource in different parts... Or maybe it would be possible and if so, I think it would make it so much easier to look back at the proxy history data. That's also where some kind of relationship graph like the ones you can get in Maltego would also be great to visualise how the different part of the website (and websiteS) interact with each other... linked to the proxy request details as well. And as I started this thread... like in Wireshark, I would love all this to be possible by just right clicking on a proxy request and just do a "follow the http stream" and would just filter out direct request relationship maybe with some level of hops you want to allow, i.e.: only in scope/same domain/sub domain/partial path. Not sure I am making much sense. I hope I am to someone here! if not I will just go back to my cave :) And if you can already achieve something similar with the current version of BURP, I would love to know how! Thanks, S.

Sylvain | Last updated: Jul 08, 2021 07:25AM UTC

Forgot to mention that I do use the colour tagging options in the proxy history. That does help, and I kind of tag the relationship but what I am requesting would be more powerful than that.

Michelle, PortSwigger Agent | Last updated: Jul 09, 2021 12:46PM UTC

Thanks for your message and for the interesting feedback. I won't make any promises at this stage but it would be good to check a few details with you to make sure I'm picturing the request properly. A stream in this context could be slightly different from TCP or HTTP streams in Wireshark. Would you see it as all the requests that went into loading a particular page or also including the steps as you click around the page? Are there any filters that you could use to identify the flow? Or would you be interested in seeing X number of requests for a particular domain after the request you had highlighted/commented and had returned to for further investigation?

Sylvain | Last updated: Jul 13, 2021 05:14PM UTC

Hi, sorry didn't get any notification there was a reply to my feature request ! Ideally I would like to see it as including the steps as I click around the page. To easily follow the flow of requests and answers generated by my interaction to a specific page/request to a server. Not sure about what filter I could use... because interacting on a page may trigger some calls to 3rd party websites, sub domains, etc. The problem I am having is when I encounter very noisy websites... sometimes the noise is numerous calls to 3rd party websites (google API, Facebook, etc). and sometimes it is just dozens/hundreds of requests to sub-domains. For the main 3rd party websites out there, like Facebook graph, etc., that's easy, I can just filter that out. but for 3dr party websites that provide scripts, I would like to keep that in... and for sub-domains I also need to keep that information. And sometimes there are some calls to website even my clients were not aware of... so retracing back why those websites were called, when, how... can be quite a slow manual task. Looking at the proxy history as a sequential list of requests/responses does not allow to easily see the relationship between all the different requests/responses, because it is just one big sequence of requests/responses. I would love to be able to see a more dynamic/intelligent representation of those requests/responses in the proxy history so we can see the different relationships between them. It would make understanding the logic of the website structure so much easier and faster. As I said to start with, not sure it is possible! :) I just had an idea of a tree with different branches where I could just dig in my proxy history and easily identify the path it took to get to a specific requests or response.

Hannah, PortSwigger Agent | Last updated: Jul 19, 2021 12:45PM UTC

Thank you for your feedback and for the level of detail you've provided! We will be discussing this internally so we can raise a feature request for this sort of functionality.

You must be an existing, logged-in customer to reply to a thread. Please email us for additional support.