Files Search Engine

Budget N/A
Bids 21
Average Bid $2802

Hello,

We are looking for someone able to create a public search engine using elastic search and nutch for crawling or the constellio system.

What we need:
1. Crawl [url removed, login to view], [url removed, login to view], [url removed, login to view], [url removed, login to view] and [url removed, login to view]
2. Use the best technique to crawl up to 1 - 2 million pages per day.
3. extract all the files name + download links
4 stock it in our database.
4. Make it "searchable" inside our search engine.

We have the global idea but looking for someone able to advice us how to realize this project like a consultant and then provide the technology to start the project.

Post a Project Like This

Looking to make some money?

  • Set your budget and the time frame
  • Outline your proposal
  • Get paid for your work

Bids on this Project

  • dennisivw Profile Picture

    dennisivw

    Zug,  Switzerland

    TBC

    PHP, Java, JSP, and Javascript

  • mmadi Profile Picture

    mmadi

    Alexandria,  Egypt

    Flowgica team has 10+ years experience in the Industry of Software Development. We target to produce high quality work. our range of service includes: Graphic and Web design Web development using most popular languages like php python ruby java and .net CMS desihn and development like wordpress joomla Mobile application development for android and iphone. E- commerce Solutions. Search Engine Optimization (SEO) and other

    PHP, C Programming, Javascript, and Python

  • linuxfreak1985 Profile Picture

    linuxfreak1985

    Karachi,  Pakistan

    turning your Ideas into GREAT ECOMMERCE PLATFORMS! Greetings! LinuxFreakz is the top freelancer based in Karachi, I have been serving Website Design / Development / Android and Server Administration jobs for past 6 years. My Expertise: 1. Wordpress Themes / Plugins Coder 2. Responsive Website Design / Bootstrap / HTML5/ CSS3 3. Woocommerce plugins / Websites 4. Android / Mobile Phone 5. Ecommerce Solutions 6. Linux Server Administration 7. INTERSPIRE / MAILWIZZ / POWERMTA 8. Email Delivery Consultant Terms of Service • 100% Customer Satisfaction • Perfect 5 star reviews!! • Good communication availability 24/7

    PHP, Java, Javascript, and Website Design

  • getveltrod Profile Picture

    getveltrod

    Chennai,  India

    Veltrod Technologies is a global software consulting company specialized in providing Mobile applications, Social media frameworks and eCommerce solutions. Leveraging best-in-class people, processes, and technologies, Veltrod provides high-quality software development and BPO services to independent software vendors and enterprises. We are specialized in developing and testing Mobile applications, Search Engine optimizations, Social Network framework development, Website design and Development.

    JSP, Javascript, Ruby on Rails, and AJAX

  • ksupport Profile Picture

    ksupport

    Madurai,  India

    I am the CEO & Co-Founder of Blaze. Personally, I do have 8+ years of PHP experience. Recently I wrote a couple of Wordpress plugins too. I have a team of 50+ employees working in my office. Complex & Customer Web Development ERP / CRM Development e-Commerce Store Development & Customizations e-Commerce Backoffice Works (order management) Cloud Migration & Monitoring Services Regards, Karthi Keyan , CEO Blaze Web Services

    PHP, ASP, Javascript, and Website Design

  • henryli2008 Profile Picture

    henryli2008

    Kunming,  China

    My name is Henry Li(Chinese name: ZIQIANG LI); I'm an experienced senior software engineer and a senior English-Chinese or Chinese-English translator,I have 14 year programming experience. located in city of KUNMING,China,and I have a software room and about 15 employee partners,including technology experts(please see our platform privided) and artist experts and english translator and American friends as technology and culture adviser and I am interested in bidding on your projects. I've recently years completed above 55 projects in Vworker.com(RAC),Now FR.com have aquired Vworker.com. So you can easily check the rating(records) of here to understand the works we have completed in recent years, that were similar to what you've described in your bid request. Recent 1-2 years,we have built about 10 working groups,include teams(small or middle company) and individuals in China andIndia,so we have enough technology and human resource power ability,and if the project meet difficulties,we have the ability of doing adjustments and overcome it.Hope we can build a virtual and honest but also some flexibility(follow our clients’ ideas) cooperations. -:) The develop platform I can adopt is: 1,Windows 2000 or 2003, Windows XP,Vista,Windows 7,IIS6.0,7.0, Classic asp3.0, CSS,Access or MS-sql to complete your project. 2, Another platform (BS STRUCTURE PROGRMME) choosing is Windows 2003 or 2000, Windows XP,Vista,Windows 7,IIS 6.0,7.0,Asp.net, Vb.net, C#.net, CSS, Access or MS-sql, Mysql; 3,Another platform (BS STRUCTURE PROGRMME) choosing is Windows 2003 or 2000, Windows XP,Vista,Windows 7,Apache,php, MS-sql,Mysql; 4,Another platform (BS STRUCTURE PROGRMME) choosing is Linux,Apache, php, AMFPHP, Mysql; 5,Another platform (BS STRUCTURE PROGRMME) choosing is Java,MySQL, JSP; 6,Another platform (CS STRUCTURE PROGRMME) choosing is Windows 2000,Windows XP, Windows 2003,Vista,Windows 7,Vb6.0, Vb.net, C#.net, Delphi,C++ builder, Access, MS-sql,Mysql; 7,Another platform is iPhone / iPad (Objective C, Xcode, Cocoa), Android (Java, Android SDK), Blackberry (J2ME, RIM), Windows, Phonegap, Sencha Touch, Games in IOS / Android (Cocos2D, Unity3D etc). 8, Another platform is Art designing,The development tools include Photoshop,Coredraw,Flash,Dreamweaver,Fireworks,Freehand,3D-max,Maya,2D,3D Animations est. 9,the database system I can adopt is, Access,MS-SQL,Mysql,ORACLE, SYBASE est; In the field of enterprise management, for example, the system of buying, selling and stock in trade, the system of finance, and to the meaning and function of the whole ERP system, I also have a deep comprehension, for the different subsystems constructed to it, for example, the buying system, the selling system,the producing system,the stockpile system, the system of finance ,CRM system, HR system, SCM system, etc. I have all made the study and analysis, either just to the function and realization of the single system or to the function and realization of the integrity of the whole system, I have also made the necessary study and analysis; besides it, to the sphere of conjunction of ERP and EC,I also made some researching works; To the Implementation of ERP, from the beginning of a excellent and executable plan to every step of strict operating, I also took a thinking and had a comprehension, to the popular problems emerged in every step, I all gave a analysis to it. This direction will be my future job direction. Besides it, in the field of OA of government department, through the successfully completion of electronic administration system, I had gained a understanding of it. To other fields of specific application software development, for I have completed many project of this field, so I have the enough confidence to the needs of study ability and adjusting ability and advantage of technology. I have much experience in being as a software system analysis engineer, I have had about two years time to being as a software system analysis engineer, Either based on the traditional analysis method of Software Life Period or the object method ,I also had the knowledge and experience; To the management of developing teams, how to encourage the member of a team, how to successfully complete your task in a high quality in time, I have got a set of methods to arrive the goal; To the tools of analysis, I am much familiar to UML and Rational Rose, partnered with other traditional tools, I had used it to completed several modeling tasks. I have built some system models to varies of software development sphere being interested to me. Hope we can build good cooperations!!! best regards Henry.Li Resume: Chinese Name: Li ziqiang. English name: Henry Li Gender: male. Age: 42 Nationality: Han Health: good. 1.Study career: (from now to the remote) University and Major: Yunnan University, computer science faculty, software major in Kumming city, Yunnan province,PR.China (1990-1994). High middle school: No3 middle school in Kumming city, Yunnan province,PR.China(1988--1990). Middle school: No1 middle school in Wenshan State ,Yunnan province,PR.China(1985--1988). Primary school: No2 primary school in Wenshan county,Yunnan province,PR.China (1979--1985). English level: Band 6 of CET (Chinese English Test) and TOEFL scores of 597 2,The description of my professional career: I was employed by one of Shanhai Songjiang electronic equipment factory as software engineer in October 1995 after graduated from university shortly, took software designing work of big style fire alarming system equipped in buildings higher then ten floors. This system adopted 8031 IC in MCS-51 sets as CPU, EPROM as program memory chip, added many extended IC, for example, RS-232, several types of triggers, digital enlarging circuit etc, forms different electronic circuit boards in different functions, and at last be made up to the whole system. The system's function is through tens to hundreds temperature inductor placed in every parts of building, transferred signal to the center machine, the center machine control different equipments, for example, sound annunciator, water spray, wrapped door, etc. to complete the functions of holding fire and get rid of fire. The operator can make program to control the whole system. The works I took was the system software programs' designing and part of the system's adjusting works, using 8086 compiling language, I had designed the programs in several types machine, accumulated rich experience. I had taken this work for two and half years. I entered One of ShenZhen,China data technology Ltd.co, as senior software engineer, this company mainly took the work of data operation. It developed the big style data operation software "Jamse Data Operation System", using C,VC++ and VB, included from scanning, recognizing, correcting, typeset, constituting and etc. a whole of automatic dealing course. I took part in the work as the software's later part's producing, modifying and maintenance. The system's difficulty located at the image's division and recognizing. Besides it, I also developed two middle style application software independently , one is "Fushang Jail computer touching screen query system", this system, related to complexed image treating and coordinate locating problem. Use part of Windows API and produced some of COM component. Another one is "JINHU Hotel menu system", in this system, I adjusted VB control very well and design the higher efficiency database structure, make the system having a high efficiency real time dealing capability. These two system both adopt ACCESS as database, WINDOWS-NT as web server, VB as develop tool. I entered One of OF ShenZhen China information technology development ltd.co (the information center of ShenZhen disabled persons federations), as web senior engineer in 2001, took the development of software used in disabled persons federation system, to now, made two big style application software system under web, one is the disabled person's remote rehabilitation system, another one is ShenZhen disabled persons federation electronic administration system. Both two systems adopted SQL-SERVER AS database server, ASP,VBSCRIPT,JAVASCRIPT as page producing tools, joined to the ShenZhen Disabled Persons website to serve to people. The disabled person's remote rehabilitation system provide the patient the remote expert diagnosing service in real time and the experts can get the real time photograph of patient, this system adopt complicated templet data structure. ShenZhen disabled persons federation electronic administration system mainly include these functions: file transfer system in ShenZhen disabled persons federation, mail system, archive system and system management. The characteristic of this system is that it's much complicated, the security requirement is higher. It successfully operated became a symbol of arriving the goal of ShenZhen disabled persons federation's "electronic disable federation". ORIGINALLY FROM JUNE,2003,BACK TO KUNMING CITY FROM SHENZHEN CITY,I BEGAN TO BUILD BROADSTRONG SOFTWARE ROOM TO NOW,MY SOFTWARE ROOM HAVE ABOUT 15 MEMBERS,ALL ARE EXPERTS TO EVERY FIELDS,OUR SOFTWARE ROOM HAVE GOT SOME PERFORMANCES INCLUDE THE PERFORMANCES IN RAC. Sincerely Best Regards Henry.Li (Chinese name: ZIQIANG LI)

    PHP, ASP, C Programming, and Java

  • navnit2011 Profile Picture

    navnit2011

    Trivandrum,  India

    I am an expert in BUSINESS AUTOMATION. I will help you to create fantastic applications and tools to MANAGE YOUR BUSINESS efficiently. I also help businesses to increase SALES and reduce COST OF OPERATION.

    PHP, Java, Javascript, and XML

  • buraqtech Profile Picture

    buraqtech

    Havelian,  Pakistan

    Buraq Technologies is an emerging software house that has its clientage from all around the globe specially from USA/UK/Europe and is specialized in web development services. We are an offshoring solutions firm with unbeatable prices,we add value with our development team by working together with the Open Source community and involving as many Open Source technologies as possible.

    Illustration, Reviews, Zen Cart, and Apache

  • AtlantaWeb Profile Picture

    AtlantaWeb

    Alpharetta,  United States

    We are Atlanta based Web Development Firm. Our experienced design team can create custom layouts/animations. The development team will design and implement the backend development if needed. We also include free site optimization and SEO consulting at the end of the project to make sure your site drives good traffic. Please visit our website at www.atlantawebexpert.com for some of our past and current projects and ask any number of references from us who will testify for our work ethic and quality. We strictly follow SDLC guidelines to make sure we deliver high quality product.

    iPhone, Android, Social Networking, and SQL

  • RajeshHub Profile Picture

    RajeshHub

    bangalore,  India

    having 6+ years experience in .Net & Apache Solr development.

    ASP, XML, .NET, and AJAX