Research Academy
My account Customers About Blog Careers Legal Contact Resellers
The Burp Suite User Forum was discontinued on the 1st November 2024.

Burp Suite User Forum

For support requests, go to the Support Center. To discuss with other Burp users, head to our Discord page.

SUPPORT CENTER DISCORD

Persistence Ballooning Files

Tyler | Last updated: Dec 13, 2023 03:36PM UTC

Hi, An extension I've built uses Montoya Persistence quite a lot. I previously ran into an issue where Burp files were ballooning because I was creating new lists every time I needed to save a new item to a list. I have since fixed that issue, but I've ran into another issue again where Burp files are 30x larger with extension data. I would like to be able to see what is actually filling the 99% of persistence. I have an export feature, and that data object is 100mb, compared to the extension data of 29gb. Is there any way to dump the persistence data? Or even, to analyse the burp file in some form of editor. Thanks for any help.

Tyler | Last updated: Dec 14, 2023 07:36AM UTC

For some added context, I've just used the get..Keys methods to dump and export every single key value to a file. It was 300mb. So I'm still at a loss as to what the other ~27.7gb are. Thank you!

Hannah, PortSwigger Agent | Last updated: Dec 14, 2023 11:54AM UTC

Hi Are you finding that your project files are ballooning in size, even when starting from a blank project file? If you save a copy of your project file excluding extension data, what size does your new file read as?

Tyler | Last updated: Dec 14, 2023 02:57PM UTC

Hi Hannah, The files start at a similar size, but slowly as the extension is used more and more, it gets larger. When the burp file without extension data is 200mb, with it is about 250-350mb. However, when the burp file is 1gb, its about 30gb. I've got a burp file here that is 29gb, and saving without extension data it is 1.61gb. On this file, if I export the data (using all the keys from persistence), the data amounts to about 300mb.

Hannah, PortSwigger Agent | Last updated: Dec 15, 2023 10:18AM UTC

It sounds like you may still be adding a lot of objects into your persisted data. Have you reviewed all the places where you're using your persisted data, to make sure you are not creating lots of new objects? Unfortunately, without taking a look at the code, this would be pretty difficult to pinpoint.

Tyler | Last updated: Dec 15, 2023 11:21AM UTC

In that case, can I ask a bit about how adding extra objects work? So for primitives and HTTP Requests, every time there is an update I would (for example) call setInteger("abc", 1) and next time I'd call setInteger("abc",2). Is this the correct way to do it? Should I be using deleteInteger("abc") before calling setInteger again? With persistedLists, I create a PersistedList object, and once I have set it with setStringList, I only ever add to that PersistedList object. Thank you

Hannah, PortSwigger Agent | Last updated: Dec 15, 2023 05:41PM UTC

Hi That sounds like you're using these in the right way. If your code is publicly available, you can share a link if you'd like, and we'll try to replicate the issues you're facing and see if we can identify what might be going wrong.

Tyler | Last updated: Dec 16, 2023 03:36PM UTC

I've been able to replicate it with some really simple code: import burp.api.montoya.BurpExtension; import burp.api.montoya.MontoyaApi; import burp.api.montoya.persistence.PersistedObject; public class PersistenceTest implements BurpExtension { @Override public void initialize(MontoyaApi api) { PersistedObject persistenceData = api.persistence().extensionData(); api.extension().setName("Persistence Test"); for (int i = 0; i < 10000000; i++) { persistenceData.setInteger("sizetest", i); } } } Setting the integer repeatedly using the same key immediately adds 100s of Mbs straight away. Even calling deleteInteger beforehand still balloons the file. I'm not sure if this is a bug or by design, but it makes it really difficult to use Persistence for anything other than preferences.

Hannah, PortSwigger Agent | Last updated: Dec 18, 2023 11:56AM UTC

Hi Thank you for the example! We've replicated your issue, and I've had a chat with the developers about the issue you're facing. This is to do with (behind-the-scenes) how we're persisting this data - essentially each time "set" is used, a new object is created. We'll be raising a feature request to improve the handling of this type of persistable data, so it doesn't have as significant an impact on disk space. If there's anything else we can help with, then please let us know.

Tyler | Last updated: Dec 18, 2023 01:47PM UTC

Appreciate you raising the feature request, I'll keep an eye out for this release!

Hannah, PortSwigger Agent | Last updated: Dec 19, 2023 09:36AM UTC

In the meantime, if you're able to provide some more information about what you're using persistence for in this manner, we may be able to suggest some workarounds for you.

Hannah, PortSwigger Agent | Last updated: Apr 04, 2024 02:54PM UTC