Program to resolve domains, their SOA , dkim , spf settings, mx records, mx host preference and the A and AAAA records of those mx hosts. The records should be stored in a database along with a time stamp of when they were resolved.
The program will continuously run and keep the records updated. Any records that change such as MX host for domain, A record for MX host, AAAA record, should be logged in a separate "history" table and the main table updated with the most recent information. Also if the domain stops resolving the domain should be logged in another table.
I would like to be able to adjust the amount of 'resolvers' or 'threads' the program will run. This way if I have to import a large set of new domains I will be able to resolve them more quickly.
I would like to be able to resolve 5 million domains in under 24 hours or about 4000/second. I have enough hardware and bandwidth to accomplish this. I also have setup my own dns resolvers.
I don't care what programming language this is done in or the database you choose to use. I do request that mongo is not an used and I do not want this written in PHP.