Job Description

Role Purpose and Overview

This role is designed for a seasoned Technical Lead specializing in Big Data and Hadoop environments. The primary focus is to drive the development and optimization of complex data pipelines, ensuring high performance, scalability, and data integrity. You will be responsible for developing, reviewing, and enhancing module-level code, designing robust replication strategies to promote data reliability, and conducting thorough root cause analyses to resolve recurring data issues. Collaboration is key, as you will work closely with both internal teams and external partners to deliver thoroughly documented, well-architected solutions that align with overarching business goals and technical standards.

This role provides a unique opportunity to lead and influence significant data engineering projects and to mentor peers, contributing to a culture of continuous improvement and shared technical excellence. You will be instrumental in enabling the organization’s data-driven strategies through innovative application of industry-leading big data technologies and architectural best practices.

͏

Key Responsibilities

• Lead comprehensive projects encompassing the design, implementation, automation, and ongoing maintenance of large-scale ETL pipelines, ensuring they meet stringent product requirements and performance benchmarks.
• Support and mentor other data engineers, facilitating their integration and growth within the team, while nurturing a collaborative, knowledge-sharing environment.
• Serve as a technical authority and subject matter expert, providing guidance in data engineering best practices and troubleshooting complex data-centric issues.
• Adopt and advocate for industry standards such as source control discipline, comprehensive code reviews, rigorous data validation, and continuous testing methodologies.
• Deliver resilient and efficient data pipelines that meet or exceed defined latency and reliability service level agreements (SLAs).
• Engage proactively with architects, software engineers, and product owners to clarify data needs and ensure alignment between technical delivery and business objectives.
• Collaborate closely with upstream data teams and governance bodies to uphold data quality standards and ensure data availability for downstream consumers.
• Develop and enhance data pipelines utilizing a variety of advanced big data technologies including Apache Spark, AWS EMR, Databricks, Kafka, NiFi, and Airflow to solve complex processing challenges.
• Lead initiatives aimed at improving automation, monitoring, and observability within data operations to promote operational excellence.
• Identify performance bottlenecks and implement optimizations using profiling tools and advanced features within technologies like Databricks to maintain cutting-edge pipeline performance.
• Continuously evaluate emerging technologies and infrastructural enhancements to refine and advance the data ecosystem.

͏

Experience & Skills Required

• Minimum of 5 years’ experience designing, building, and operating robust end-to-end data pipelines with a strong emphasis on data accuracy, integrity, and quality management.
• Comprehensive understanding of data engineering methodologies and firm grounding in software engineering principles. Candidates must possess a bachelor’s degree in a technical or quantitative discipline such as Computer Science, Engineering, Mathematics, or Statistics; advanced degrees are highly valued.
• Exceptional communication skills — both verbal and written — and proven ability to foster effective collaboration and teamwork in dynamic, matrixed, and geo-distributed environments.
• Proven adaptability and eagerness to embrace challenging projects, learning opportunities, and cross-functional teamwork across diverse teams and locations.
• Deep familiarity with agile methodologies and test-driven development, ensuring development agility and product quality.
• Essential Technical Competencies:
• Extensive hands-on experience with big data platforms like Apache Spark and cloud-based data solutions such as Databricks.
• Strong proficiency in advanced programming and scripting languages including SQL, Python, and Scala.
• Sound knowledge of data pipeline architecture (ETL/ELT), data modeling techniques including star schema designs, and data warehousing concepts.
• Expertise in orchestration and streaming technologies such as Apache Airflow, Kafka, and NiFi to enable high-throughput real-time data flows.

͏

Mandatory Skills and Qualifications

• Proven expertise in Hadoop Cloudera ecosystems, including design, deployment, and troubleshooting within enterprise environments.
• Experience working with distributed computing frameworks and managing data storage solutions optimized for big data workloads.
• Strong analytical skills to perform root cause analyses and implement sustainable solutions for complex data issues.
• Demonstrated ability to lead technical teams delivering high-impact projects on time and to specification, with a commitment to continuous delivery and improvement.
• Commitment to professional growth and staying current with emerging trends in big data and cloud data technologies.

͏

About Wipro Technologies

At Wipro Technologies, we are pioneering the future of digital and data transformation across multiple industries. As a leading end-to-end digital services provider, we are committed to empowering our workforce to design their own reinvention and build the next generation of technology solutions. Our culture fosters bold ambitions supported by collaboration, innovation, and purpose-driven work.

We believe in cultivating an inclusive environment where diverse perspectives thrive, fueling creativity and outstanding results. If you are passionate about evolving your career while contributing to impactful projects that harness cutting-edge technologies like Hadoop Cloudera, Spark, and cloud platforms, then Wipro is the place for you.

Join us and become part of a team dedicated to excellence, continuous learning, and driving transformational change across the technology landscape. Together, we create meaningful impact, powering businesses with data-powered insights and intelligent solutions.

Experience Required: 5-8 Years

Mandatory Skills: Hadoop Cloudera