Simple Database Model and WP-Admin pages to manage the data.
(Project completed in 7 days max)
What we need: *Additional information in attached document.
Custom Database and WP-Admin pages to manage data
1. Database/tables schema created to manage our Data Lists "we will provide example data file, also please see below for our list of rows for the primary table". Would be no more then ~ 3-5 tables but I leave it up to you experts to decide the best model.
2. Ability to manage our data from custom pages in wp admin. I see this as a tab in the WP Admin with a few pages or sub-items of the tab to perform the operations mentioned below (Import, Export, Reporting...)
3. Ability upload data lists and perform data associations "see the image at the bottom" - we will be able to explain in more detail and are open to recommendations. Our data list files for import will be large files between 5-100megs.
4. Ability perform Queries on the data list tables to pull targeted lists based on custom search criteria and have ability to export and save the data to comma delimited .csv files with max rows of 100,000 "so if 1,000,0000 rows returned then there would be 10 .csv files created". Would be optimal to export the files "if more then 1) to .zip file. The search functionality will need to have ability to perform Custom Select Statments or use a Search Box and Select Boxes to determine what to search the data by "I can provide specific information for this".
5. Ability to add IP/Domain and Email Addresses to a Blacklist Table in the Database and all queries will reference this to not include the record if the record has either a IP/Domain or an Email Address associated with the Blacklist.
6. WP Admin Page to display information/report about the Data in the Database (Total Rows of Data), (Last Import Performed), (Last Export Performed)
NOTE: Import/Export operations need to display information about the successful or errored rows during an import and or export.
• 1,000,000 rows imported
• 840,000 rows successful
• 160,000 rows failed to insert (Link to view error report)
1. Database needs to have certain fields verified before allowing an insert
• IP address and Domain “regex to validate domain and or IP” make domain standardized with “http://” i.e. "[url removed, login to view]" would become "[url removed, login to view]" and IP standardized.
• datetime would need to be a standard format for all data
2. Database needs to verify "upon data upload" the IP/Domain and Email Addresses against a blacklist table. Need a page to insert records into the blacklist manually. Do not want to pull any records that ip/domain is in the blacklist table.
3. Database model needs to be able to handle 100’s of millions of records and all queries need to be optimized for efficiency.
*Additional information in attached document.