CFR File Scraper, Downloader, and File Processing 11-4-2021

File Scraper, downloader, and file processing. YOU MUST READ THE "CFR PROJECT [login to view URL]" BEFORE YOU BID. This project consists of three parts: 1. Spider through a website and capture every link to be downloaded into a txt file 2. Download and naming all files that result from the spidering. The files will be downloaded into 50 folders with the following format. THE YEAR COMES FROM THE CURRENT YEAR. The 1cfr, 2cfr etc is the title number: 1cfr2021 2cfr2021 3cfr2021 through 50cfr2021 2. Format each file downloaded to a specific format 3. You must create three process logs: a. For each file found in the spider process. b. Files downloaded and not downloaded c. Files processed and not processed 4. You must create a graphics file log, recording the file location for each graphic ink found within any file when it is processed. See the [login to view URL] file. The [login to view URL] file will contain the links to each graphic found in the file processing and contains a list of links SEE THE ATTACHED "CFR PROJECT [login to view URL]" FOR THE DETAILED PROJECT REQUIREMENTS 5. You must be able to accommodate both regular regulations and the Appendix sections 6. Some of the regulations have different levels of spidering. This program must be able spider to as many levels deep the individual text or appendix is located. 7. The footer and the graphics link locations must be changeable in a config file (see [login to view URL] file). 8. The program must have the ability to run multiple copies of the software at the same time. 9. The program must run from an EXE file on a Windows 10/11 computer. 8. We require the source code as well as the finished program at the end of the project 10. Attached are raw data documents and finished documents to be used as a guide. 11. There are 225,000 files that are located, downloaded, and processed. Please review the information carefully before you provide a bid, as there will be no changes to the contract price once we accept your bid. Please view the attached file for a sample of what the file format will be when completed. There are both regular and appendix text in this sample. We provide all funds in a Freelancer escrow account. You must complete this project within 30 days (or less) You must reply to all communications within 24 hours