QlikView – how many records can you have in a document?Posted: March 12, 2012
Triggered by a discussion last week, I just had to test how many records that could be stored in one QVW-document. The result is amazing!
My laptop is an one year old Lenovo T410. 8 GB ram 2×2.27 GHz dual processors (Intel core i5 430M). So it is not a monster. 6.100 MB RAM were available for my test. I decided that I wanted to create a script that generated one table with one field that could contain numbers from 0 to 9. Then I made some tests to estimate the maximum numbers of records that could be expected and started the test.
I ended with 5.368.709.120 number of records!
I saved the document and created a new document where I did a binary load from the first QVD. I loaded the 5.4 billion records in 48 seconds, that’s 111.848.106 records per second! I had to try the same with Excel PowerPivot, but it crashed, even when I delimited the load to 1.000.000.000 records. It was exactly the same data and data source. I wrote this blog in about 30 minutes – I could have loaded 200.000.000.000 records instead (200 billion) – I need more RAM.