write 2 python web crawlers using scrapy framwork to read wikipedia data

Implement two web crawlers in python using the scrapy (1.0.5) framework 1) Get the full list of countries and territories from here [login to view URL] write a. Country / territory name b. Wikipedia URL c. Status (Membership) d. Dispute status e. Further information f. Polling date into a mariadb based database db schema: id(auto-incement), createdate(timestamp), all other fields are type text 2) Get the list of URLs in 1b) and crawl each one of the countries websites to extract information of each one of them: a. Abstract b. VCard data: i. Name ii. URL for flag iii. URL for emblem iv. Motto v. Anthem vi. URL to location on globe vii. URL to map viii. Capital(s) 1. Name 2. URL ix. Official language(a) 1. Name 2. URL x. Religion(s) 1. Name 2. URL xi. Demonym(s) 1. Name 2. URL xii. Government 1. Name 2. URL xiii. Establishment(s) 1. Name 2. Date xiv. Area 1. Total km2 2. Water km2 xv. Population 1. Total estimate 2. Date of counting / estimate xvi. GDP 1. Total 2. Per capita xvii. HDI index 1. Total 2. Rank xviii. Currency 1. Name 2. 3 letter code xix. TimeZone(s) 1. Name 2. Deviation from GMT 3. URL to timezone xx. Driving on left or right? xxi. Calling code(s) xxii. ISO code xxiii. Internet TLD(s) c. Date of polling Write the data above into a mariadb based database db schema: id(auto-incement), createdate(timestamp), all other fields are type text In case of multiple entries (e.g. languages) write a comma separated list in the db text field. Make sure the original text is comma free. Requirements: - Running on Ubuntu 14.04 lts (x64) standard installation (scrapy 1.0.5 installation [login to view URL]) - Mariadb 5.5 - Needs to run failsafe with correct results for all countries – especially for countries with several entries for capitol, timezones, languages etc Copyright: - All code belongs to employer Delivery: - 2 web crawlers with pipeline into MariaDB - Code with comments / documentation to be maintained - After uploading the full code, I will run it on my system and proof-read code before payment - No milestones, payment in full after successful test