solr indexing expert needed for some advice java urgent

This project received 8 bids from talented freelancers with an average bid price of $415 USD.

Get free quotes for a project like this
Employer working
Skills Required
Project Budget
$250 - $750 USD
Total Bids
Project Description

We need urgent help with SOLR indexing SPEEDS:

We are indexing MYSQL tables to SOLR. We are not indexing documents, we are indexing rows only and many are only IDs / Numbers. The data size is small, but rows are almost 80 BILLION total.

We have tried 3 separate ways. I need urgent help with someone who has done this type of thing before and got very fast speeds.

We are currently manage 50k rows per second (from tables in mysql with no JOINS) (using DIH)
and 15k rows per second (from tables in mysql WITH JOINS). (using DIH)

50k CONSTANT we can just accept. but JOINS we need much better (40k per sec)

Currently we need 80 BILLION rows indexing over 25 days. We have 1 large server ONLY:

192gb ram, SSD drives, 2x hexacore CPU.

Please contact me urgent if you have solution and can help.


Looking to make some money?

  • Set your budget and the timeframe
  • Outline your proposal
  • Get paid for your work

Hire Freelancers who also bid on this project

    • Forbes
    • The New York Times
    • Time
    • Wall Street Journal
    • Times Online