In Progress

Remove duplicate records from a huge MySQL databases

Hi,

I have many mysql databases and each database contains many tables. The whole databases and tables contains TENS OF MILLIONS of records, but the problem is there are many duplicates among these records. So I built a php script to remove the duplicates, but it is very slow. Anyway, you need to read the following:

1- Your main job is to find a fastest way to remove the duplicates from all tables in each database, and you have to give me a feedback about the speed.

2- I prefer to get 1 (.exe) file written in perl or any good language, the file should get instructions from me to remove duplicates from a specified database or from all the databases. Ex: if I enter the name of DBs (db1,db2 ...etc), it can be start removing duplicates from these specified databases, or if I enterd (*), it can go and remove duplicates from all the databases that you specified in the code in advance.

3- I'll give you a PHP script to build the databases and tables automatically, and sample file of millions of records with a script to insert these records to the databases.

4- After completing the project, I need to test it using Teamviewer or any other way to make sure that it is meet my requirements and to check the speed too.

5- $30 - $50 is my max budget.

6- When you bid, please send me your thoughts about how we can get this job done.

Skills: Apache, MySQL, Perl, PHP, Shell Script

See more: php mysql remove duplicates script, how to find a job, give me a job, teamviewer 3, Remove, remove from, find a name, databases, remove duplicates mysql, database remove duplicates, sample perl, php file remove, perl sample, php remove file, php databases, remove script, advance mysql, mysql teamviewer, mysql feedback, perl code sample, read exe, duplicate file, check duplicate, max build script, mysql remove duplicate

About the Employer:
( 10 reviews ) ankra, Iraq

Project ID: #1103017

Awarded to:

ciachn

Hello please check PMB.

$40 USD in 2 days
(7 Reviews)
4.7

20 freelancers are bidding on average $44 for this job

stels88

Please check PMB.

$50 USD in 2 days
(151 Reviews)
6.3
linuxfreak1985

Please check your PM Box

$50 USD in 1 day
(75 Reviews)
5.8
gasparch

Please see PM

$49 USD in 1 day
(44 Reviews)
5.6
kommandant

i can remove dublicate records

$80 USD in 2 days
(24 Reviews)
4.8
triadb

Hi, please check PM

$50 USD in 7 days
(12 Reviews)
4.0
amsak

Placing bid

$50 USD in 7 days
(10 Reviews)
3.8
dakunil

please see PMB

$35 USD in 2 days
(3 Reviews)
3.2
rapunzelme

I have few algorithm which can do the job. Please check PMB for details.

$30 USD in 2 days
(2 Reviews)
2.4
deatul

Dear, kindly check PM.

$35 USD in 0 days
(4 Reviews)
2.4
croca

Hi my friend, i will tell you my secret for free, i have 20 years of experience as oracle DBA, when doing huge delete on big tables, the best you can do is the opposite: 1- create new-table like the table you need to More

$30 USD in 0 days
(0 Reviews)
0.0
msajaynath

Sir check pm.

$30 USD in 3 days
(2 Reviews)
0.0
senthilm1985

hi, i will do ur task, please give what u want to remove, thank you

$30 USD in 2 days
(0 Reviews)
0.0
guptav21

i have done this same thing several time using perl

$35 USD in 1 day
(0 Reviews)
0.0
q0l8ueu

may i try it? but i want to see your database and data...

$50 USD in 3 days
(0 Reviews)
0.0
Gyanda

I will be opting for C program. It will create a hash table with all the records. Those recordes which causes hash clash will be removed. This will not take any extra tablespace and will not impact database perform More

$55 USD in 5 days
(0 Reviews)
0.0
Ashwini19830112

we have done this type many times. plz see your PMB

$50 USD in 2 days
(0 Reviews)
0.0
rahimpk

Hi Sir, I have been working in Perl/Oracle/PLSQL in morethan 5.5 years. I got your requirement. Kindly allocate this project to me. Regards, Rahim P.K

$50 USD in 2 days
(0 Reviews)
0.0
shyamsang

I will able to create binary file(C++/Perl) for your requirement..

$50 USD in 2 days
(0 Reviews)
0.0
Aleksandrow

Hello, Please, check the PM

$30 USD in 1 day
(0 Reviews)
0.0
Tijssen

Hi, With over 8 years experience with PHP and MYSQL, I suppose I can fix your problem. How can we go on so I can take a look in your database, structure etc. Regards, Jeroen

$30 USD in 1 day
(0 Reviews)
0.0