EXPLORE

Data Processing Java Python Software Architecture SQL
Profile cover photo
User Avatar

Shahzad Ali K.

@shzdalikhan

0.0
0

0.0

0.0

0%

AI & Web Scraping • Backend • Data & Cloud Expert

$15 USD / Hour

・

Flag of

Pakistan (9:25 PM)

・

Joined on December 7, 2024

$15 USD / Hour

・

I am a Senior Backend Engineer specializing in AI-powered automation, cloud-native backend systems, and advanced web scraping. With 15+ years of experience, I design and deliver scalable, resilient and production-ready solutions across data engineering, APIs, microservices, scraping pipelines, and DevOps automation. I build enterprise-grade web scraping systems capable of handling Cloudflare protection, dynamic content, region-locked data, and large-scale extraction. My scrapers use Scrapy, Playwright, proxy rotation, browser fingerprinting, CAPTCHA solving (2Captcha), automated retries, failover logic, session management and resume-from-last-state capabilities. I also build real-time dashboards using Flask/FastAPI with live logs, metrics, job monitoring, and one-click scraping controls. On the backend side, I develop high-throughput microservices, distributed systems, and event-driven architectures using Python, Scala, Java, Akka, and FastAPI. I integrate APIs, build data-heavy automation, and deliver fully containerized applications with CI/CD pipelines. As a data engineer, I create robust ETL/ELT pipelines using Airflow, Databricks, Spark/PySpark, DBT, Lakehouse/Medallion architecture, Delta Lake, and streaming/batch workloads. I help clients modernize legacy systems to high-performing, cloud-optimized platforms. On the DevOps and cloud side, I work extensively with Docker, Kubernetes, Terraform, Helm, GitHub Actions, JenkinsX, and cloud-native logging/monitoring stacks such as Prometheus, Grafana, FluentD. 🧩 Key Capabilities 🔹AI-powered web scraping engines (Scrapy, Playwright, proxies, CAPTCHA solving) 🔹Backend engineering (APIs, microservices, distributed systems) 🔹Data engineering (ETL/ELT, Spark, Airflow, Databricks, DBT) 🔹Dashboard development (Flask/FastAPI) for automation & monitoring 🔹Cloud & DevOps (Kubernetes, Docker, Terraform, CI/CD) 🔹Database & storage integrations (PostgreSQL, Hive, Cassandra, Delta Lake) 🔹Production-grade architecture, testing & deployment ⚙️Technical Skill Set ▸Languages: Python, Scala, Java, SQL ▸Frameworks: Scrapy, SeleniumBase, Flask, FastAPI, Akka ▸Big Data: Spark/PySpark, Databricks, Delta, Airflow, Kafka, Hive, Parquet, Avro, Kubeflow, Strimzi ▸DevOps: Docker, Kubernetes, Helm, Terraform, GitHub Actions ▸Databases: PostgreSQL, DB2, Cassandra ▸OS & Tools → Linux (Debian/Ubuntu), Bash, SonarQube, PMD, Checkstyle ▸Monitoring: Prometheus, Grafana, FluentD ✨Message to Clients I focus on long-term, reliable technical partnerships. You can expect: ✔ On-time delivery ✔ High-quality, maintainable architecture ✔ Clear communication ✔ Free post-delivery support ✔ Strong technical ownership Let’s collaborate to create a solution that is scalable, robust and built to last.

Verifications

Portfolio

Portfolio

13731245
13731215
13731205
13731199
13731178
13712882
13731245
13731215
13731205
13731199
13731178
13712882
Reviews
0 Reviews

No reviews to see here!

Experience

Senior Freelance Data, Backend & Automation Engineer

Dec, 2019 - Present

•

6 years, 5 months

AWA Labs (Freelance Agency)

Dec, 2019 - Present

•

6 years, 5 months

⭐Web Scraping & Automation ⭐AI-Powered Data Extraction ⭐Custom API Development ⭐Data Pipeline & ETL Engineering ⭐Cloud Deployment (AWS, GCP, DigitalOcean) ⭐Python Backend Development ⭐Web Dashboard Development (Flask, FastAPI) ⭐RAG-based Chatbot Development ⭐n8n Automation Workflows ⭐Binance Trading Bot Development

Flag of

Lahore, Pakistan

Dec, 2019 - Present

•

6 years, 5 months

Data Engineer

Apr, 2017 - Nov, 2019

•

2 years, 7 months

Cloud Card, inc

Apr, 2017 - Nov, 2019

•

2 years, 7 months

📌EOD Batches Re-engineering ▸Replaced legacy Java + SQL OLAP workflows with distributed pipelines ▸Developed Spark + Airflow + Cassandra architecture running on Kubernetes ▸New system processed 100× more data without impacting the OLTP workload ▸Implemented Ceph storage integration & automated CI/CD via Jenkins

Flag of

Lahore, Pakistan

Apr, 2017 - Nov, 2019

•

2 years, 7 months

i2c, inc

Jan, 2005 - Mar, 2017

•

12 years, 2 months

Principal Software Engineer

Aug, 2013 - Mar, 2017

•

3 years, 7 months

📌T-API (Issuer Processing Microservices) ▸Designed microservices for issuer-side PCI financial operations ▸Implemented ISO-8583 integration for Visa & MasterCard, plus REST for closed-loop partners ▸Migrated monolith to Akka + Kafka + Cassandra + Kubernetes ▸Increased throughput from 100 TPS → 3000 TPS

Flag of

Lahore, Pakistan

Senior Software Engineer

Dec, 2008 - Jul, 2013

•

4 years, 7 months

📌Cards Vault (PCI DSS / GDPR) ▸Designed a secure vault for storing cardholder SAD data ▸Built large-scale issuance and inventory automation workflows ▸Ensured PCI DSS & GDPR compliance 📌Fast Settle (Real-time Settlement System) ▸Built near-real-time settlement pipeline with configurable merchant hierarchies ▸Developed flexible commission calculation workflows ▸Designed multi-method payout execution modules

Flag of

Lahore, Pakistan

Software Engineer

Jan, 2005 - Dec, 2008

•

3 years, 11 months

📌API Integrations ▸Integrated SafeNet HSM (EMV, contactless, magnetic stripe) ▸Built KYC integrations with Experian, LexisNexis & Equifax ▸Integrated Load Centers (GreenDot, MoneyGram, BHN) ▸Integrated payment processors (CyberSource, Beanstream, BPAY)

Flag of

Lahore, Pakistan

Jan, 2005 - Mar, 2017

•

12 years, 2 months

Education

National University of Computer and Emerging Sciences

2000 - 2004

•

4 years

Bachelor's in Computer Sciences

Flag of

Pakistan

2000 - 2004

•

4 years

Registered Users

Total Jobs Posted

Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)

Copyright © 2026 Freelancer Technology Pty Limited (ACN 142 189 759)