Completed

RSS crawler for different languages and categories fetched from database

I have mysql database where we need to feed it with news from RSS websites.

Considering the RSS links are in database, linked to categories and the categories are linked to countries so for example:

-USA

-- Politics

--- [url removed, login to view]

---- [url removed, login to view]

---- [url removed, login to view]

---- [url removed, login to view]

--- [url removed, login to view]

---- [url removed, login to view]

---- [url removed, login to view]

---- [url removed, login to view]

etc...

We need to create a well clean and clear pure PHP script based on PHP7 where it will read the RSS and insert them into the database in the table article. I will provide you with the database script where we have there the following tables:

===Database news

== Table structure for table article

|------

|Column|Type|Null|Default

|------

|//**article_id**//|int(255)|No|

|category_id|int(11)|No|

|language_id|int(11)|No|

|rss_id|int(100)|No|

|article_title|tinytext|No|

|article_date_time|datetime|No|

|article_details|text|No|

|article_image|varchar(255)|No|

|article_video|varchar(255)|No|

|article_page_name|varchar(255)|Yes|NULL

|article_importance_level|int(10)|No|1

== Table structure for table category

|------

|Column|Type|Null|Default

|------

|//**category_id**//|int(10)|No|

|category_name|varchar(255)|No|

|language_id|int(4)|No|

|category_color|varchar(10)|No|

|category_order|int(3)|No|1

== Table structure for table language

|------

|Column|Type|Null|Default

|------

|//**language_id**//|int(4)|No|

|language_name|varchar(100)|No|

|language_flag|varchar(255)|No|

|language_country_name|varchar(255)|No|

|language_order|int(11)|Yes|NULL

== Table structure for table rss

|------

|Column|Type|Null|Default

|------

|//**rss_id**//|int(100)|No|

|rss_link|tinytext|No|

|category_id|int(10)|No|

|language_id|int(4)|No|

|rss_frequency_minutes|int(4)|No|

Once we agree I will send you mysql script to create the database on your pc.

The script should crawl as well the video links, the images links, article title, description, etc.. (mainly everything in the table "article" should be filled by your script).

1- First script will be called from the cron job which will loop over the languages/countries (country and language are the same to us) and call the second script and pass for it the language_id as parameter.

2- Second script will be called from via the first script, get the language_id as parameter, fetch relevant categories from database, and foreach category, fetch the relevant RSS links and send each RSS link to a third script as request param.

3- Third script will take the rss link (should accept all kind of links like .xml and .rss) and will crawl the data, filter them and insert them into article (that's the objective here)

The scripts must be optimized for speed, comments should be on the code for easy editing later on, not too many nested function to avoid losing connectivity of the task, no frameworks should be used, just simple PHP classes, no complex OOP required, MVC is not needed.

The project should be done within couple of days from the moment we agree on.

I would like to see sample of the code via a hangout with the candidate.

Thank you.

Skills: MySQL, PHP, Software Architecture, XML

See more: software write mq4, software write chip epson, useful software write book, software write web specs, free software write user guide, database design web crawler, software write edid, free software write company profile, software write websites idea, rss page, software write book images, google maps database categories, joomla 500 server error rss page, software write books, add rss page, access database web page filter, software write protection, free software write book, joomla adsmanager database categories, joomla different front page registered, project database categories subcategories, software write book, different myspace page setups, google maps php database categories, web crawler copying web page

About the Employer:
( 2 reviews ) Dusseldorf, Germany

Project ID: #12228533

Awarded to:

bijaya47

Hello Sir Greetings of the day! I have gone through the project specification. I have prior experience with similar types of websites. I am a developer with 4 yrs of experience with wordpress, woo commerce, corep More

€238 EUR in 4 days
(137 Reviews)
6.6

13 freelancers are bidding on average €231 for this job

ITLove007

How Are You, Sir. I have mobile app( Android, IOS) , php and backend skill. I have experience with Social app, Dating app, Chatting app, Video Streamming, IPTV app, Uber style App , Restaurant/Food Deliver app, phot More

€631 EUR in 3 days
(44 Reviews)
7.0
toseef3

Hello I have read your requirements and I am willing to do this project for [url removed, login to view] are an experienced team based in Pakistan, having 9+ years of experience in Enterprise, web and smartphone app development. We have More

€147 EUR in 3 days
(64 Reviews)
7.3
ATLASFSD

A proposal has not yet been provided

€200 EUR in 1 day
(68 Reviews)
6.2
webperfection123

EXPERT WORDPRESS DEVELOPER with 5 YEARS EXPERIENCE:: Thanks a lot for sharing the details regarding the project. I have completely checked the requirements and fully comfortable to work on your job . I am 100% Sure More

€188 EUR in 5 days
(83 Reviews)
5.8
satvainfotech

Hello, We are expertise Team of Professionals having experience of more than 12+ years in IT Fields. We have very good exposure in .NET, WCF/WPF, MVC 4+, MSSQL, LAMP, PHP, MySQL, Wordpress, CakePHP etc.., We ar More

€277 EUR in 3 days
(15 Reviews)
5.6
FutureTechno1

Hi there, I’d like to be considered for your job position. I’m a Software Developer with a strong background developing web application. I can turn your requirement in a way that represents your brand and appeals t More

€155 EUR in 3 days
(14 Reviews)
5.3
€388 EUR in 3 days
(10 Reviews)
5.2
hirakhan2

Hi,  I have review project details and would like to discuss with you regarding the project. Kindly let me know when available. Thank you.

€194 EUR in 7 days
(14 Reviews)
4.7
ArkssTech

Hello I have reviewed your job description as well and very confident to handle this project perfectly. I am expert in Web designing/developing.I believe that I can be the best productive candidate for this require More

€155 EUR in 3 days
(17 Reviews)
4.2
pthreeinfotech

Hi...... I am Pritesh and I represent Pthreeinfotech. I can complete your project on time. I have worked on similar projects to what you are looking for, and I am confident I can exceed your expectations. I read throug More

€200 EUR in 3 days
(26 Reviews)
4.3
accreteinfo

Dear, The only confusion is with "article_importance_level" how will it be determine. Please share your convenient time to discuss project in details on hangout. We can finish this within 2-4 days. Hoping for yo More

€111 EUR in 3 days
(1 Review)
1.9
syncnologies

Hello Sir/ Ma’am, We have pool of dedicated team consisting of certified developers capable enough to handle your project exactly according to requirements and within said time frame. As per shared project de More

€122 EUR in 3 days
(2 Reviews)
0.6