Stack Exchange network consists of 183 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.
I added a comment on the BitBucket issue. By the way, I was wrong about my total run time, since I didn't take standby into consideration. I think the total active import time was actually closer to 7-10 hours.
I have a hunch that you could speed up the program significantly by utilizing multiple cores for SQL and Solr access since those appear to be the bottlenecks, but so far I haven't been able to figure it out. (I'm not very familiar with SQL, Solr, or Python multiprocessing, unfortunately.) Have you looked at all at Solr's DataImportHandler? They claim to be able to import Wikipedia's 40GB XML dump in about 50 minutes, which might be useful for speeding up Stackdump.
Sam, this is amazing! Thank you so much for making this. I can't believe how professional this product feels, from the instructions to the tools to the website. Import of SO took 21.5 hours on my 2.3GHz Haswell i7 Macbook — slightly annoying, but tolerable if you do it on your internal drive, since you can put your computer to sleep and seamlessly resume later. (Strangely, row processing seemed to slow down towards the end of the import.) Out of curiosity, how long did this take to make?