Senior Web Data Engineer (Remote) Job at MAP SSG, Remote

RWd4N1V2bkVUTHpTVkxEMlphb1I5NDNF
  • MAP SSG
  • Remote

Job Description

Summary

We need someone with 4+ years of experience in web data engineering, API integration, or similar roles, who has strong experience with Python libraries (requests, web scraping frameworks (Scrapy, BeautifulSoup, Selenium), and modern browser automation tools (Playwright, Puppeteer). You should be comfortable designing and maintaining robust data collection pipelines from public APIs and web sources, and have a deep understanding of web technologies like HTML, CSS, JavaScript, REST APIs, and GraphQL. You should have up-to-date knowledge of proxy management, session handling, and TLS techniques. Bonus points if you have experience with Rust and/or Go web scraping frameworks for performance, and familiarity with distributed systems and job queues.

What you'll do :
  • Design and maintain robust data collection pipelines from public APIs and web sources.
  • Build and optimize web scraping systems for publicly accessible content.
  • Transform and normalize collected data into structured formats for downstream processing.
  • Monitor data quality, collection performance, and compliance with platform guidelines.
Responsibilities :
  • Design and maintain robust data collection pipelines from public APIs and web sources.
  • Build and optimize web scraping systems for publicly accessible content.
  • Transform and normalize collected data into structured formats for downstream processing.
  • Monitor data quality, collection performance, and compliance with platform guidelines.

Requirements :

  • Either or both of
    • Strong experience with Python libraries (requests, web scraping frameworks (Scrapy, BeautifulSoup, Selenium), modern browser automation tools (Playwright, Puppeteer)
    • Strong experience with Rust and/or Go web scraping frameworks for performance
  • Deep understanding of web technologies: HTML, CSS, JavaScript, REST APIs, GraphQL
  • Understanding of network protocols, DNS, and web infrastructure
  • Familiarity with data formats and serialization (JSON, Parquet, Protocol Buffers)
  • Knowledge of proxy management, session handling, and TLS techniques
  • Strong background in data validation, cleaning, and transformation pipelines
  • 4+ years of experience in web data engineering, API integration, or similar roles
  • Experience working in cross-functional teams with strong communication skills

Bonus requirements :

  • BS/MS in Computer Science, Engineering, or equivalent experience
  • Experience with distributed systems and job queues (Redis, Celery, Amazon SQS)
  • Knowledge of web scraping practices and compliance frameworks
  • Familiarity with Kubernetes for scalable data collection
  • Experience with database systems for storing and indexing collected data
  • Public GitHub repositories demonstrating web scraping or API integration projects

Benefits :

  • Medical, Dental, Vision, Life Insurance, STD and LTD Plans
  • FSA - Medical and Dependent Care
  • EAP and wellness programs
  • Productivity stipends
  • 13 Paid Holidays
  • Unlimited PTO
  • Flexible work environment - 100% remote
  • Bi-annual company/team meetups
  • 401(k) plan with employer matching contributions
  • Annual review for salary raises
Salary
$150-200K

Job Tags

Remote work, Flexible hours,

Similar Jobs

BrightStar Care of Indianapolis

RN Medicaid Nurse with Experience in Veterans and Insurance Services - Hamilton County Job at BrightStar Care of Indianapolis

 ...for certain Clients)* Part of a team with RN Oversight* Home Health or Facility shifts...  ..., preferred.* Possess exceptional nursing assessment skills.* Analytic/Critical thinker.* Licensed driver with valid automobile insurance* Ability to work on weekends, nights and... 

Pioneer Desert Manufacturing

Appliance Service Technician Job at Pioneer Desert Manufacturing

 ...Diagnose, repair and install major residential appliances (refrigerators, washers/dryers, ovens, etc.) in shop and field settings. Key responsibilities Perform electrical, mechanical and refrigeration diagnostics with multimeters & gauges Order and replace faulty... 

Pernini Legal, LLC

Real Estate Finance Associate Attorney (Jr.) - Dallas or Denver Job at Pernini Legal, LLC

 ...REAL ESTATE FINANCE - JUNIOR ASSOCIATE ATTORNEY Locations: Dallas, TX | Denver, CO HYBRID...  ...AmLaw 100 firm , is seeking a junior-level Associate Attorney to join their...  ...engaging with some of the largest structured commercial real estate finance transactions in the... 

Empire Merchants North

Merchandiser - Distributor of Wine & Spirits / Syracuse Job at Empire Merchants North

 ...North is the premier wine and spirits distributor in Upstate New York with roots going back...  .... Ability to multi-task, work independently and/or within a team, pays attention to...  ...Empire Merchants North is an equal opportunity employer and, as such, affirms in policy... 

IT Concepts, Inc

SharePoint Developer Job at IT Concepts, Inc

 ...Department of Veterans Affairs-End User Operations and Monitoring (ESOM)! We are seeking a highly skilled and motivated Senior SharePoint Developer to join our team in supporting the VA ESOM Business Operations. In this critical role, you will be responsible for...