Speeding up database processes


We have a database with some large tables, and we're having trouble getting all the processes involved to all run quickly [url removed, login to view] have three different queries for pages that we need to run more quickly, balancing that with one import script. For instance, adding some indexes to our largest table speeds up the page loads, but slows down the import. We're looking for an approach that will let all the processes run as quickly as possible.

We also recently changed our tables from myISAM to innoDB, which has caused some other problems to arise. Now we're getting locking errors that we didn't get before. We're also getting another strange error: we have a Perl import script that reads from several data files into our database. There's one section of code that checks whether a given record already exists, and if not, inserts it. Now, early during the import script we usually get an error that we've tried to insert a given record twice (the table has a unique index using two of the columns). This may also be related to some changes we made to the Perl script, but given the nature of the changes made, that doesn't seem likely.

We'd like someone to take a look at our database, page queries, and import script and advise on whether we need to take a different approach, or how we can improve our existing one.


Skills: MySQL

See more: database problems, processes, error exists page, mysql error 2013, record checks, related database, perl script data mysql, insert data database mysql, mysql queries using, page loads different, mysql database slows, import large mysql database, mysql database errors, large mysql database, perl table import database, database queries mysql, import large mysql, adding database instance mysql, instance mysql, large mysql table, perl import mysql script, getting large files database, mysql large database, perl script mysql import, large import mysql

About the Employer:
( 12 reviews ) new york, United States

Project ID: #4141225

9 freelancers are bidding on average $339 for this job


Dear, I've understood your requirements. I'll do it. Regards.

$250 USD in 5 days
(51 Reviews)

Hi. I'm have many experience with MySQL. I can do this project for you very fast and highest quality. Thanks

$250 USD in 5 days
(31 Reviews)

Hello Sir. Please check PMB

$300 USD in 5 days
(41 Reviews)

Dear. I'm interested with your project. Do you mind to choose me? I'll do it for you. I'll wait for your choosing. Thanks.

$250 USD in 5 days
(34 Reviews)

I'm ready to start now, please contact me.

$250 USD in 2 days
(9 Reviews)

Hi! You can checkout my specific experience in database trouble shooting and speed up in my past projects. I can work on your servers or set up a similar environment on my server to checkout your problems and rec More

$500 USD in 10 days
(13 Reviews)

Hi, I'm interested that project - please check PM for more details.

$500 USD in 3 days
(8 Reviews)

10+ years perl programmer, 6+ years mysql dba.

$500 USD in 10 days
(1 Review)

Maybe i can assist u, more detail in pm.

$500 USD in 7 days
(0 Reviews)

Hi Brian, I have extensively worked on MySQL performance tuning with InnoDB as well as MyISAM engines. I can take this forward. Thanks Abhijit

$250 USD in 5 days
(0 Reviews)