Job Description
Role Purpose
The purpose of the role is to support process delivery by ensuring daily performance of the Production Specialists, resolve technical escalations and develop technical capability within the Production Specialists.
͏
Role: Lead Data Engineer
Location: San Diego( Onsite only )
Lead Data Engineer JD:
Role Summary
We are seeking a highly skilled Lead Data Engineer with deep, hands-on experience in building large-scale data ingestion pipelines, real-time streaming solutions, and high-performance data processing systems. The ideal candidate excels at writing clean, efficient code, refactoring complex systems, improving scalability and performance, addressing production issues, and delivering reliable data solutions across cloud platforms such as AWS, Azure, or GCP.
Key Responsibilities
- Design, build, and maintain high-volume data ingestion and processing pipelines for batch and real-time workloads.
- Implement and optimize real-time streaming pipelines using platforms such as Kafka.
- Develop scalable data solutions using Databricks, PySpark, Python, and SQL.
- Perform core refactoring to modernize and optimize existing pipelines and data services.
- Build robust, fault-tolerant pipelines capable of large-scale, high-throughput data processing.
- Write unit tests, automate validation, and ensure high code quality and reliability.
- Integrate pipelines into CI/CD workflows to streamline and automate deployment processes.
- Identify, troubleshoot, and fix production issues, ensuring system reliability and stability.
- Address performance bottlenecks and implement improvements for scalability, throughput, and efficiency.
- Work extensively across AWS, Azure, or GCP cloud environments and cloud-native data services.
- Design and orchestrate end-to-end pipelines using workflow and orchestration tools.
- Collaborate with Data Scientists and BI Engineers to deliver clean, analytics-ready datasets.
- Communicate complex technical topics clearly to non-technical stakeholders.
Required Qualifications
- 8+ years of hands-on data engineering experience building and maintaining large-scale data systems.
- Proven experience with high-volume data ingestion, ETL/ELT, and real-time data processing.
- Strong expertise with Kafka or similar streaming technologies.
- Advanced proficiency in Databricks, PySpark, Python, and SQL.
- Experience with core refactoring, improving code maintainability and system performance.
- Demonstrated ability to design and build scalable, low-latency data pipelines.
- Strong skills in debugging, performance optimization, and pipeline tuning.
- Hands-on experience with at least one major cloud platform: AWS, Azure, or GCP.
- Experience writing unit tests and integrating solutions with CI/CD pipelines.
- Strong problem-solving skills and excellent communication abilities.
Experience: 5-8 Years .
The expected compensation for this role ranges from $80,000 to $160,000 .
Final compensation will depend on various factors, including your geographical location, minimum wage obligations, skills, and relevant experience. Based on the position, the role is also eligible for Wipro's standard benefits including a full range of medical and dental benefits options, disability insurance, paid time off (inclusive of sick leave), other paid and unpaid leave options.
Applicants are advised that employment in some roles may be conditioned on successful completion of a post-offer drug screening, subject to applicable state law.
Wipro provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state, or local laws. Applications from veterans and people with disabilities are explicitly welcome.
Reinvent your world. We are building a modern Wipro. We are an end-to-end digital transformation partner with the boldest ambitions. To realize them, we need people inspired by reinvention. Of yourself, your career, and your skills. We want to see the constant evolution of our business and our industry. It has always been in our DNA - as the world around us changes, so do we. Join a business powered by purpose and a place that empowers you to design your own reinvention.