This post investigates the performance of unmarshalling an XML document to Java objects using a number of different approaches. The XML document is very simple. It contains a collection of Person entities.
There is a corresponding Person Java object for the Person entity in the XML
...
and a PersonList object to represent a collection of Persons.
The approaches investigated were:
There is a corresponding Person Java object for the Person entity in the XML
...
and a PersonList object to represent a collection of Persons.
The approaches investigated were:
- Various flavours of JAXB
- SAX
- DOM
In all cases, the objective was to get the entities in the XML document to the corresponding Java objects. The JAXB annotations on the Person and PersonList POJOS are used in the JAXB tests. The same classes can be used in SAX and DOM tests (the annotations will just be ignored). Initially the reference
implementations for JAXB, SAX and DOM were used. The Woodstox STAX parsing was then used. This would have been called in some of the JAXB unmarshalling tests.
The tests were carried out on my Dell Laptop, a Pentium Dual-Core CPU, 2.1 GHz running Windows 7.
Test 1 - Using JAXB to unmarshall a Java File.
Test 1 illustrates how simple the progamming model for JAXB is. It is very easy to go from an XML file to Java objects. There is no need to get involved with the nitty gritty details of marshalling and parsing.
Test 2 - Using JAXB to unmarshall a Streamsource
Test 2 is similar Test 1, except this time a Streamsource object wraps around a File object. The Streamsource object gives a hint to the JAXB implementation to stream the file.
Test 3 - Using JAXB to unmarshall a StAX XMLStreamReader
Again similar to Test 1, except this time an XMLStreamReader instance wraps a FileReader instance which is unmarshalled by JAXB.
Test 4 - Just use DOM
This test uses no JAXB and instead just uses the JAXP DOM approach. This means straight away more code is required than any JAXB approach.
Test 5 - Just use SAX Test 5 uses no JAXB and uses SAX to parse the XML document. The SAX approach involves more code and more complexity than any JAXB approach. The Developer has to get involved with the parsing of the document.
The tests were run 5 times for 3 files which contain a collection of Person entities. The first first file contained 100 Person entities and was 5K in size. The second contained 10,000 entities and was 500K in size and the third contained 250,000 Person entities and was 15 Meg in size. In no cases was any XSD used, or any validations performed. The results are given in result tables where the times for the different runs are comma separated.
TEST RESULTS
The tests were first run using JDK 1.6.26, 32 bit and the reference implementation for SAX, DOM and JAXB shipped with JDK was used.
JDK 1.6.26 Test comments
JDK 1.6.26 + Woodstox test comments
JDK 7 + Woodstox test comments:
implementations for JAXB, SAX and DOM were used. The Woodstox STAX parsing was then used. This would have been called in some of the JAXB unmarshalling tests.
The tests were carried out on my Dell Laptop, a Pentium Dual-Core CPU, 2.1 GHz running Windows 7.
Test 1 - Using JAXB to unmarshall a Java File.
Test 1 illustrates how simple the progamming model for JAXB is. It is very easy to go from an XML file to Java objects. There is no need to get involved with the nitty gritty details of marshalling and parsing.
Test 2 - Using JAXB to unmarshall a Streamsource
Test 2 is similar Test 1, except this time a Streamsource object wraps around a File object. The Streamsource object gives a hint to the JAXB implementation to stream the file.
Test 3 - Using JAXB to unmarshall a StAX XMLStreamReader
Again similar to Test 1, except this time an XMLStreamReader instance wraps a FileReader instance which is unmarshalled by JAXB.
Test 4 - Just use DOM
This test uses no JAXB and instead just uses the JAXP DOM approach. This means straight away more code is required than any JAXB approach.
Test 5 - Just use SAX Test 5 uses no JAXB and uses SAX to parse the XML document. The SAX approach involves more code and more complexity than any JAXB approach. The Developer has to get involved with the parsing of the document.
The tests were run 5 times for 3 files which contain a collection of Person entities. The first first file contained 100 Person entities and was 5K in size. The second contained 10,000 entities and was 500K in size and the third contained 250,000 Person entities and was 15 Meg in size. In no cases was any XSD used, or any validations performed. The results are given in result tables where the times for the different runs are comma separated.
TEST RESULTS
The tests were first run using JDK 1.6.26, 32 bit and the reference implementation for SAX, DOM and JAXB shipped with JDK was used.
| Unmarshall Type | 100 Persons time (ms) | 10K Persons time (ms) | 250K Persons time (ms) |
| JAXB (Default) | 48,13, 5,4,4 | 78, 52, 47,50,50 | 1522, 1457, 1353, 1308,1317 |
| JAXB(Streamsource) | 11, 6, 3,3,2 | 44, 44, 48,45,43 | 1191, 1364, 1144, 1142, 1136 |
| JAXB (StAX) | 18, 2,1,1,1 | 111, 136, 89,91,92 | 2693, 3058, 2495, 2472, 2481 |
| DOM | 16, 2, 2,2,2 | 89,50, 55,53,50 | 1992, 2198, 1845, 1776, 1773 |
| SAX | 4, 2, 1,1,1 | 29, 34, 23,26,26 | 704, 669, 605, 589,591 |
- The first time unmarshalling happens is usually the longest.
- The memory usage for the JAXB and SAX is similar. It is about 2 Meg for the file with 10,000 persons and 36 - 38 Meg file with 250,000. DOM Memory usage is far higher. For the 10,000 persons file it is 6 Meg, for the 250,000 person file it is greater than 130 Meg.
- The performance times for pure SAX are better. Particularly, for very large files.
| Unmarshall Type | 100 Persons time (ms) | 10K Persons time (ms) | 250K Persons time (ms) |
| JAXB (Default) | 168,3,5,8,3 | 294, 43, 46, 43, 42 | 2055, 1354, 1328, 1319, 1319 |
| JAXB(Streamsource) | 11, 3,3,3,4 | 43,42,47,44,42 | 1147, 1149, 1176, 1173, 1159 |
| JAXB (StAX) | 30,0,1,1,0 | 67,37,40,37,37 | 1301, 1236, 1223, 1336, 1297 |
| DOM | 103,1,1,1,2 | 136,52,49,49,50 | 1882, 1883, 1821, 1835, 1822 |
| SAX | 4, 2, 2,1,1 | 31,25,25,38,25 | 613, 609, 607, 595, 613 |
- Again, the first time unmarshalling happens is usually proportionally longer.
- Again, memory usage for SAX and JAXB is very similar. Both are far better
than DOM. The results are very similar to Test 1.
- The JAXB (StAX) approach time has improved considerably. This is due to the
Woodstox implementation of StAX parsing being used.
- The performance times for pure SAX are still the best. Particularly
for large files.
| Unmarshall Type | 100 Persons time (ms) | 10,000 Persons time (ms) | 250,000 Persons time (ms) |
| JAXB (Default) | 165,5, 3, 3,5 | 611,23, 24, 46, 28 | 578, 539, 511, 511, 519 |
| JAXB(Streamsource) | 13,4, 3, 4, 3 | 43,24, 21, 26, 22 | 678, 520, 509, 504, 627 |
| JAXB (StAX) | 21,1,0, 0, 0 | 300,69, 20, 16, 16 | 637, 487, 422, 435, 458 |
| DOM | 22,2,2,2,2 | 420,25, 24, 23, 24 | 1304, 807, 867, 747, 1189 |
| SAX | 7,2,2,1,1 | 169,15, 15, 19, 14 | 366, 364, 363, 360, 358 |
- The performance times for JDK 7 overall are much better. There are some anomolies - the first time the 100 persons and the 10,000 person file is parsed.
- The memory usage is slightly higher. For SAX and JAXB it is 2 - 4 Meg for the 10,000 persons file and 45 - 49 Meg for the 250,000 persons file. For DOM it is higher again. 5 - 7.5 Meg for the 10,000 person file and 136 - 143 Meg for the 250,000 persons file.
- No memory analysis was done for the 100 persons file. The memory usage was just too small and so it would have pointless information.
- The first time to initialise a JAXB context can take up to 0.5 seconds. This was not included in the test results as it only took this time the very first time. After that the JVM initialises context very quickly (consistly < 5ms). If you notice this behaviour with whatever JAXB implementation you are using, consider initialising at start up.
- These tests are a very simple XML file. In reality there would be more object types and more complex XML. However, these tests should still provide a guidance.
- The peformance times for pure SAX are slightly better than JAXB but only for very large files. Unless you are using very large files the performance differences are not worth worrying about. The progamming model advantages of JAXB win out over the complexitiy of the SAX programming model. Don't forget JAXB also provides random accses like DOM does. SAX does not provide this.
- Performance times look a lot better with Woodstox, if JAXB / StAX is being used.
- Performance times with 64 bit JDK 7 look a lot better. Memory usuage looks slightly higher.
Thanks for the write up! Ever try using HTMLCleaner with XPath to parse? I wonder its performance compares.
ReplyDeleteNice post!
ReplyDeleteHave you came across VTD-XML?
The following link has some performance comparison:
http://vtd-xml.sourceforge.net/benchmark1.html
The Woodstox numbers are, down to the millisecond, the same as the RI ones.
ReplyDeleteI don't think this can be correct...
Correct Henning. I have updated with correct times.
ReplyDeleteThanks for sharing the test results, every result set I've seen (including my own) has always shown SAX to destroy the competition. I think your analysis of the test results is flawed though.
ReplyDeleteFirst off, the JDK is caching things by evidence of the high results on the first run, then drastically reduced times on every test after that. Maybe should have created a different data file for each cycle of the test so that it wouldn't be able to cache. For example, first test would process a 250K persons file, next test cycle process a 250K animals file, then a 250K cars file, then a 250K train file, etc. Would ensure the JDK doesn't skew your test results, but still allows you to see the difference in the JDK load/preparation times of the first cycle vs the following cycles.
Second, you're not taking full advantage of SAX because you're storing every Person object in a List. The entire point to using SAX is so that you do not store the entire contents of the XML into memory, so you're getting a lot of benefits from SAX already, but the memory footprint of SAX should be near zero if you do it right, which should reduce the times even further for SAX while not requiring any memory to do the work. If you needed 1 MB of memory to process the 100 persons file, then you should still only need 1 MB of memory to process the 250,000 persons file.
Lastly, I think your statement is wrong when you say times for SAX are only "slightly" better and only for large files, so you shouldn't use SAX unless working with those large files. Even with the test results you show, SAX is at least 4x faster (or more) than every other type of parsing, and that is on the small files. The larger files SAX is a minimum of 2x faster. That sure seems like its more than "slightly" better. If I can be guaranteed a 2x to 4x improvement in speed by writing a bit more logic into my code and require no memory footprint to do it, I'd do that any day unless I'm absolutely sure I will not be processing more than a few records.
Hi Sams,
ReplyDeleteThanks for your comments. In response:
1. The time isn't always much higher the first time. In some cases there is very little difference. For example, JAXB(Streamsource), JAXB (StAX) using JDK 1.6.26 using 10K Persons.
But yes I agree - in general - the JDK looks like it is doing someting special the first time. I'd argue that's a find in this analysis not a flaw. The other tests scenarios you mention sound interesting. There's quite a lot of possible test permutations here - impossible to cover everything. What is most important - I'm sure you agree - is that you cover the scenarios that are most relevant for your architecture. I will try to cover more permutations in future posts.
2. Yes the SAX test is storing persons in a list. But that was intentional. As stated in the blog post the objective was "to get the entities in the XML document to the corresponding Java objects". That's a different test scenario to
when you are only interested in some of the entities in the XML. In this test, we are interested in *all* entities. That means they have to be stored somewhere. It might have being more complete if the SAX handler had a getter but that won't change any of the times.
3. By "slight improvement" regarding SAX what I meant was the absolute difference rather than
the relative difference. If you add log4J to a method which is just a simple accessor you
will probably double the execution time of it. This would just be a simple boolean check to see if log4J was enabled but still it takes time. If you remove the log4J you may cut down the execution time of that method by half but in the overall scheme of things you are probably not getting any significant improvement. This is because both the check to see if log4J and the actual returning of a variable take very little time.
Except for cases of unmarshalling very large files, in most cases, the differences between JAXB and SAX were around 20 milliseconds. This absolute difference would be even less on a production server. Remember the tests were just on my laptop. What I am suggesting is that 20 milliseconds isn't going to get noticed, unless you are doing a very time critical application.
One has to remember what your architecture is trying to do. It's extremly unlikely that it is just parsing XML for the sake of it. Usually something else is going to happen, for example data is about to be stored in a database or sent to the Web Tier. Whatever it is, if there is a performance problem with the use case it would be more beneficial to consider everything. Checking your database indexs or zipping your javascript may yield a more noticeable benefit.
My argument is that in most cases the progamming model of JAXB will be beat the performance of SAX. However, if your requirements is for saving milli-seconds than its SAX.
Hi,
ReplyDeleteHow you are measuring the memory ?
By using memorymx bean or some different way !!!
If you are using by this mxbean then do you consider the gc also?
Please include XStream also here. Because it is also a common api.
ReplyDelete