Open

Postgres Query Generation over Millions of Rows of Oil and Gas Data (XPATH a Plus)

Project Description is too long for this post. Please read attached word doc for full description.

We would like to buy a block of 80 hours doing PostgreSQL queries over Oil & Gas data. We are hovering at around 200 million rows of data that feed multiple user interfaces. All of this data needs a ton of cleaning and curation.

The PostgreSQL stuff will not be hard to do. Lot of case when, concat, substrings… just moving data from poor formats to good formats. For example, we may get a VARCHAR that is 1/1/2017 and need to convert it to date. Two second change. But we have hundreds of these date fields. Sometimes in a table with 1MM rows, we will have one badly formatted date – 1&1/2017. Getting that one bad date fixed is not the goal. We have to work around it to get the other 999,999 into date format. We need that VARCHAR in date format. And sometimes, the dates come in as 1/2017. Or some come in as MM/YYYY and others (in same field) are in MM/DD/YYYY format. You get the picture!

The easy part will be PostgreSQL. The hard part? Oil & Gas data is not written in lay speak. It’s a very technical industry… tubing, casing, tvd, formations… little of it will make sense. Of course, if you know O&G, huge bonus, but it’s not necessary or expected for this project.

We will interpret all the O&G stuff (for you) and send you simple sentences on what each c_ and m_ field should look like. You will then build the queries for each table.

80 hours could just be the beginning. We are already working on a Phase II. We would like to build aggregation queries around all 200 million rows for “highlights” we can report on a webpage. This would entail running row counts/sums/etc. for a ton of fields and even some basic equations. We would update these queries every night and have a web page that reports all the “macro” changes to our data sets each day. This would be used to educate users on how vast and complex the data is behind the UIs.

Finally, if you know XPATH, that’s a huge bonus for this project. We can engage you beyond the 80+ hours of PostgreSQL work to running our Web Scraper to go get more data. More data? More PostgreSQL cleaning and curation work. XPATH is not mandatory though. We prefer someone who is super comfortable writing queries. You will need that comfort while you are working with the Oil & Gas terminology.

Skills: Database Administration, PostgreSQL, Software Development, XML

See more: google api chart using php mysql, data cleansing oil gas chennai, using google checkout dnn html module, kuwait oil gas companies details, oil gas copywriter, name list oil gas construction companies kuwait, kuwait oil gas contracting companies, flash website oil gas company, system administrator oil gas, oil gas flash template, oil gas projects kuwait, oil gas gnucash, oil gas contractors kuwait, design build oil gas yahoocom, adobe flex google finance chart, website wwwglobalphonecallcom currently process transactions using google checkout, test loop diagram oil gas, using google docs math, electronics instrumentation oil gas, using google docs freelancer, using google earths kml asp php, service calculate driving distance using google api, mechanical systems design oil gas, google finance chart creator, graphical identification authentication using webcam

About the Employer:
( 4 reviews ) Oklahoma City, United States

Project ID: #15634240

19 freelancers are bidding on average $1328 for this job

sphinxsolution

Hi, An upscale infrastructure of 8000 Square feet accommodating a team of 150+ in Pune, India. Our flexible onsite and offshore model helped us in scaling our reach & serving global clientele. Relevant Skills and Expe More

$1500 USD in 30 days
(1 Review)
6.3
tudiptechnology

Hi there, We have proven track record of delivering Java based web applications (DropWizard, Jersey, Struts, JSF/RichFaces) with different front end libraries (AngularJS/JQuery) and template engines. Relevant Skills a More

$1400 USD in 21 days
(2 Reviews)
5.1
$1250 USD in 10 days
(2 Reviews)
2.4
infoistic

Hi there I understand that you need an expert in database, specially PostgreSQL. I finished a telecom audit project in postgres which needed about 100TB space containing all call detail record of everyday call. It nee More

$2000 USD in 10 days
(5 Reviews)
2.2
$1250 USD in 20 days
(1 Review)
2.2
TakaAlex

Hi Can complete your project perfectly. Always Keep trust and High Quality Review me in my portfolio Thank you Relevant Skills and Experience Database Administration, PostgreSQL, XML Proposed Milestones $1250 USD - More

$1250 USD in 20 days
(1 Review)
2.4
Darwin21015

Hello, I would like to work in your project. I have two years of experience in portgresql with complex queries. For me it would be a real challenge to help them. you can write me i am allways here. Habilidades y exp More

$1250 USD in 10 days
(2 Reviews)
1.8
kraziel78

Hi, I work with PostgreSQL on a daily basis, creating reports for the different areas of the company that I work for. I'll be glad to help you cleaning your data. Jonathan.- Relevant Skills a More

$1444 USD in 25 days
(0 Reviews)
0.0
vivirain

Hi there, Hope you’re doing well. I’m interested in the job and would like to work on it over 80+ hours or more. I have around 10 years of SQL experiences and also have sound knowledge of Postgres SQL, as well as X More

$1111 USD in 10 days
(0 Reviews)
0.0
AzzkaNoor

Greetings. I just have a visual over your project's details and feel privileged to tell you that this correlates with my domain of expertise. Please message me as i will like to ask a few questions. Relevant Skills an More

$1250 USD in 20 days
(0 Reviews)
0.0
anitath123

Hello Thank you for your posting job. I have checked your job in detail and am confident on your job. I have full experience in IOS/ANDROID app development and ever made a lots of mobile apps Relevant Skills and Exper More

$1250 USD in 20 days
(0 Reviews)
0.0
reedsystems

Hi, how are you today? I read the description of your request, I get the complexity of the task. Cleaning and Querying records from a database can be a demanding task based on how the dirty the data is. I have wo More

$800 USD in 22 days
(0 Reviews)
0.0
Lultu

The project sounds quite interesting and a long one. The problem area is cleaning so many-many rows perfectly. Better to handle it using stored procedures and some sql jobs. for future have such jobs. Relevant Skills More

$1444 USD in 30 days
(0 Reviews)
0.0
skavras

I'd like participate the project. I have enough free time and good relevant experience to do this. My working hours are within 6AM-8PM UTC, so have normal intersection with USA business time. Relevant Skills and Exper More

$1333 USD in 20 days
(0 Reviews)
0.0
widuIT

Hi There, This is Nico, I was invited by Stella to the project. Already took a look at the attached document and even I´m more proficient working with MySQL, at the end of the day, and based on what you wrote, it is More

$1764 USD in 30 days
(0 Reviews)
0.0
philiphankinson

I have worked as a SQL (relational) database consultant for over 25 years. Also, I have well over seven years using PostgreSQL and GIS data (the majority of my PostgreSQL experience includes using the PostGIS spatial e More

$1111 USD in 20 days
(0 Reviews)
0.0
ajoy3freelancer

I have 13 years experience in development so complex large queries are not uncommon to me. I agree I have no knowledge in oil and gas domain, but if I get proper explanation of the tables I can work Relevant Skills an More

$1333 USD in 10 days
(0 Reviews)
0.0
Yamy31

I guarantee you a first class service with the quality and in the time it deserves by putting at your disposal all my knowledge and years of practice. Habilidades y experiencia relevante I have more than 6 years of ex More

$1244 USD in 18 days
(1 Review)
0.0
$1250 USD in 20 days
(0 Reviews)
0.0