Read large xml files java

It does XML parsing all the time, and tries really hard to be robust to syntax errors. Usually, memory is the bigger issue. Stack Overflow for Teams — Collaborate and share knowledge with a private group.

Create a free Team What is Teams? Collectives on Stack Overflow. Learn more. Asked 6 years, 4 months ago. Active 6 years, 4 months ago.

Viewed 13k times. Need help regarding the above issue. Thanks in advance. Improve this question. Have not reached solution yet but looks like this may be helpful: andreas. Open source library for parsing parts of XML documents at a time, designed for similar situations to this. The tricky part is processing the information delivered by the parser - and to help you with that, we need to know what you want to do with the data.

Could you provide me with a sample and an XSD? Add a comment. Active Oldest Votes. Improve this answer. Kasper van den Berg 2, 15 15 silver badges 29 29 bronze badges. YoYo YoYo 4 4 silver badges 15 15 bronze badges.

You will need to develop such tree collection code on top of a streaming parser. Also for searching for a tag you still want to use a descent SAX based parser, just to handle those oddball situations - where maybe you have more than one kind of 'account'.

SAX offers you good context tracking. I am curious though how easy it would be to switch live between the 2 - sounds like something I want to do. So xml2j streams to POJO objects keeping only a single or few in memory.

Is this assumption correct? Actually it never switches to tree mode. Instead on every context change it switches handlers. Each handler knows how to process that particular element type. This approach was suggested by Robert Hustead as explained here: slideshare. Possibly change the buffer size in the BufferedReader?

In any case, what counts as a "long time"? It's 3 million lines. Why output it to the screen in the first place? Who's going to be able to read that? Campbell Ritchie.

Dave Tolls wrote:. That may be the explanation for the slow execution. Liutauras Vilda. With pull parsing, the client controls the application thread and can call methods on the parser when needed, which perfectly works for our case. The other advantages of the pull-based approach and comparisons between the different available parsers can be found in the below-mentioned document.

Here are some code snippets:. Note : The 2 suffixes used in code, i. Parsing XML document and saving the results in the Request object. Changing data in XML file and writing to the new file. Processing All Files parallelly.

For the purpose of parallel execution, we have utilized ExecutorService provided by Java with the number of initial threads configurable. For synchronization, we have used CountDownLatch. The sample implementation for this is provided in the below code snippet:. Once all the processing is done in XMLTask, we call latch.

With latch.