SOLR large data indexing help

CLOSED
Bids
8
Avg Bid (USD)
$1434
Project Budget (USD)
$250 - $750

Project Description:
Hi,

We deal with huge solr data (huge rows - 10 billion, but only small size - 200GB) We are trying to index it to solr in fastest way from mysql DB.

The solr importer is custom and works very well, and mysql works very well and is not the bottleneck. The issue is is solr does not commit the documents fast enough. it hangs and then stops after 100 million rows.

We can easily get 100,000 rows per second from all parts, apart from the actual submit / index to solr. More information can be found here:

https://issues.apache.org/jira/browse/SOLR-5075?page=com.atlassian.jira.plugin.system.issuetabpanels:changehistory-tabpanel

We need urgent help and advice from an expert who has used solr to index these sort of speeds / data before.

Thanks

Skills required:
Apache Solr, Big Data, Database Administration, Java, MySQL
About the employer:
Verified
Public Clarification Board
Bids are hidden by the project creator. Log in as the employer to view bids or to bid on this project.
You will not be able to bid on this project if you are not qualified in one of the job categories. To see your qualifications click here.


Hire programmedf
$ 666
in 10 days
$ 1111
in 15 days
Hire Bramfeld
$ 650
in 10 days
Hire angan
$ 5555
in 60 days
Hire pardeepkaursonia
$ 500
in 10 days
Hire niketkumar
$ 666
in 10 days
Hire i7informatics
$ 777
in 3 days
$ 1546
in 30 days