Closed

solr indexing expert needed for some advice java urgent

We need urgent help with SOLR indexing SPEEDS:

We are indexing MYSQL tables to SOLR. We are not indexing documents, we are indexing rows only and many are only IDs / Numbers. The data size is small, but rows are almost 80 BILLION total.

We have tried 3 separate ways. I need urgent help with someone who has done this type of thing before and got very fast speeds.

We are currently manage 50k rows per second (from tables in mysql with no JOINS) (using DIH)

and 15k rows per second (from tables in mysql WITH JOINS). (using DIH)

50k CONSTANT we can just accept. but JOINS we need much better (40k per sec)

Currently we need 80 BILLION rows indexing over 25 days. We have 1 large server ONLY:

192gb ram, SSD drives, 2x hexacore CPU.

Please contact me urgent if you have solution and can help.

Thanks

Skills: Apache Solr, Big Data, Java, MySQL

See more: java indexing, solr java, urgent java, urgent - java, total advice, urgent solution, mysql tables rows, indexing solr, java using much cpu, solr urgent, constant contact expert, java data mysql, ssd solution, need java expert help, mysql indexing using solr, ids using java, java fast, constant contact java, domainmanuk, java expert needed, java large numbers, numbers java, mysql expert needed, expert solr, cpu mysql

About the Employer:
( 41 reviews ) dalts, United Kingdom

Project ID: #4685542

8 freelancers are bidding on average $415 for this job

IMSeriousBidder

Hello, let me know what 3 separated ways you used? Bing

$618 USD in 3 days
(76 Reviews)
6.9
twalters84

Hey There, I have a lot of experiencing indexing SOLR documents from Coldfusion, which is Java based. I have never dealt with an amount of data that large. However, I do have an idea on how fix the problem. More

$333 USD in 30 days
(74 Reviews)
6.2
shenchilang

Experienced java developer.

$315 USD in 7 days
(41 Reviews)
5.6
sushiljain11

Greetings, I have 5 years of industrial experience in Solr,Lucene,Java MongoDB, MySQL, PHP and I am Oracle Certified Professional Java programmer. Currently, I am working as Search & Optimization consultant and my More

$350 USD in 3 days
(20 Reviews)
4.7
javaguruji

Worked on Solr extensively specailly on DIH. I have indexed DB(Oracle) with 80 million records with 256 columns.

$333 USD in 7 days
(0 Reviews)
0.0
sekarj

I have good experience in Java, MySQL indexing and also worked with Solr some time back I have done applications reading a large number of records, sorting them and storing as xml format and hence confident of solv More

$300 USD in 7 days
(0 Reviews)
0.0
Uttam25

i will do it for u..thnx

$555 USD in 8 days
(0 Reviews)
0.0
dbconsultingvw

I might be able to help you out.

$515 USD in 14 days
(0 Reviews)
0.0