In Progress

Simple Program To Remove Duplicate Domains From Text File

I need someone to write a simple program which can do the following -

Scan a .txt file which contains 1 URL per line. Check to see whether the domain name of any line is duplicate, and if it is, remove all instances of the URL.

For example - the program would check each line for [url removed, login to view] (tld being the domain extension such as .com, .org, .net, .info etc) - if two lines containing [url removed, login to view] exist, then the ENTIRE LINE will be deleted (not just the [url removed, login to view]).

Each URL will consist of more than just the domain and tld, for example URL's may consist of [url removed, login to view] however it's important that if [url removed, login to view] is found to exist on 2 or more lines, then the ENTIRE line is removed, not just the [url removed, login to view] portion.

Quite simply I want a program that will check a text file with a list of URL's, find any duplicate domains, and then remove the ENTIRE URL of any lines containing that domain and tld.

It should be something simple like: "if [url removed, login to view] exist in 2 or more lines then remove all lines containing word1.word2".

Further Examples:

Let's say the .txt file looks like the following -

[url removed, login to view]

[url removed, login to view]

[url removed, login to view]

[url removed, login to view]

This program should remove line 1 & line 2, since the domain and tld are the same. The text file should be left looking like -

[url removed, login to view]

[url removed, login to view]

Notice how [url removed, login to view] and [url removed, login to view] were NOT removed since there was only ONE instance of each domain and tld. If a domain and TLD only exist ONCE, then they should NOT be removed.

If you need further instructions please message me.


Skills: C Programming, PHP, Visual Basic

See more: php remove text url, php remove line duplicate, net remove duplicate domains, program find, in line program, how to write on line, how to check a php program, find someone to write, find a program, php remove duplicate domains, remove duplicate domains, duplicate domain remove text file, url remove duplicate domain, remove duplicate domains php, scan duplicate url, vbscript list duplicate lines text file, remove duplicate text line, php program check dupliacte lines text file, file remove line, scan text files, php duplicate url delete, check duplicate lines file vbscript, find duplicates txt file, find duplicate text file, duplicate text remover

About the Employer:
( 2 reviews ) Sydney, Australia

Project ID: #443014

Awarded to:


Hi.. Am expert in developing Visual basic Application... Please check PM...

$30 USD in 1 day
(12 Reviews)

15 freelancers are bidding on average $33 for this job


I will create siimple PHP script to parse text file. And output result according to your rules. Can be done today.

$30 USD in 1 day
(1258 Reviews)

Hello,Please refer your [url removed, login to view] you.

$35 USD in 1 day
(86 Reviews)

Please check PMB.. jexplorer

$30 USD in 0 days
(153 Reviews)

Please read my message.

$30 USD in 1 day
(68 Reviews)

Hello sir, I can do this for you and give you the script in few hours. Please select me and I will make it. Thanks

$44 USD in 0 days
(39 Reviews)

Hello. I will do it in one hour. Sergey S.

$30 USD in 1 day
(11 Reviews)

Dear Sir, Please check your PMB. Thank you, Best Regards, GWT

$30 USD in 0 days
(2 Reviews)

Hi, I can get this done for you today in VB.

$30 USD in 0 days
(2 Reviews)

Hello, I have lot of experience in programming. I can create the application exactly in accordance to your description using Java. You will be 100% satisfied with the work. Thanks.

$50 USD in 3 days
(3 Reviews)

I've got realy good skills in C/C++, also have deep intret in C/C++. Done alot of projects in C.

$30 USD in 1 day
(0 Reviews)

I am expert in regular expression. Please allow me help you.

$30 USD in 3 days
(0 Reviews)

I can do it in C++, VB or perl whatever language you prefer in one day. Ready to start as soon as you make your choice.

$40 USD in 1 day
(0 Reviews)

I'm a PHP professional, but it's not needed to be a professional to do this project. If I would be awarded, I'll give you a PHP script in 30 minutes or less afrer I realize that I've awarded for this job. Actually, it More

$30 USD in 0 days
(0 Reviews)

very easy job. It can be done within a couple of hours or less. Regards.

$30 USD in 0 days
(2 Reviews)