Job Description
Hadoop / ETL Developer and PySpark
Location – Dallas, TX / Atlanta, GA / Cleveland, OH / Pittsburgh, PA (Day 1 Onsite)
Job Description:
We are seeking a highly experienced Senior Big Data & DevOps Engineer with 8+ years of professional experience in HDFS, Hive, Impala, PySpark, Python, and DevOps automation tools such as uDeploy and Jenkins. This role is responsible for managing end-to-end data operations, including HDFS table management, ETL pipeline development, multi-environment codebase governance, platform upgrades, and production support.
The ideal candidate will have strong expertise in Linux system operations, Big Data ecosystem tools, and experience with incident/change management using ServiceNow. This role plays a key part in ensuring the stability, scalability, and efficiency of enterprise data platforms while enabling seamless development-to-production workflows.
Key Responsibilities:
Big Data Platform Operations
• Design, manage, and optimize HDFS directories, tables, and partitioning strategies.
• Implement and enforce data retention and lifecycle policies across large datasets.
• Administer Hive and Impala environments, ensuring high availability, performance tuning, and security compliance.
ETL Development & Data Engineering
• Develop scalable ETL pipelines using PySpark, Hive, and Python.
• Build reusable frameworks for data ingestion, transformation, and aggregation.
• Optimize job performance through query tuning, resource management, and parallelization.
DevOps & Environment Management
• Maintain and promote code across DEV, QA, UAT, and PROD environments.
• Develop and support CI/CD pipelines using Jenkins and uDeploy for automated deployments.
• Perform environment upgrades, patching, and dependency management aligned with release schedules.
Linux & Infrastructure Operations
• Execute Linux administration tasks including performance tuning, disk management, and scripting (Bash/Python).
• Troubleshoot cluster-level issues including node failures, job errors, and distributed system anomalies.
Change & Incident Management
• Drive incident resolution and change execution using ServiceNow workflows.
• Conduct root cause analysis (RCA) for critical issues and implement preventive solutions.
• Ensure compliance with ITIL processes for change, incident, and problem management.
Collaboration & Technical Leadership
• Partner with data engineers, developers, DevOps teams, and business analysts to ensure operational excellence.
• Mentor junior engineers and contribute to technical leadership across the Big Data ecosystem.
• Document operational procedures, troubleshooting guides, and architectural decisions for internal knowledge sharing.
Required Qualifications:
• Bachelor’s degree in Computer Science, Information Technology, or related field.
• 8+ years of experience in Big Data engineering and DevOps practices.
• Advanced proficiency in HDFS, Hive, Impala, PySpark, Python, and Linux.
• Proven experience with CI/CD tools such as Jenkins and uDeploy.
• Strong understanding of ETL development, orchestration, and performance optimization.
• Experience with ServiceNow for incident/change/problem management.
• Excellent analytical, troubleshooting, and communication skills.
Nice to Have:
• Exposure to cloud-based Big Data platforms (AWS EMR).
• Familiarity with containerization (Docker, Kubernetes) and infrastructure automation tools (Ansible, Terraform).
͏
Do
- Oversee and support process by reviewing daily transactions on performance parameters
- Review performance dashboard and the scores for the team
- Support the team in improving performance parameters by providing technical support and process guidance
- Record, track, and document all queries received, problem-solving steps taken and total successful and unsuccessful resolutions
- Ensure standard processes and procedures are followed to resolve all client queries
- Resolve client queries as per the SLA’s defined in the contract
- Develop understanding of process/ product for the team members to facilitate better client interaction and troubleshooting
- Document and analyze call logs to spot most occurring trends to prevent future problems
- Identify red flags and escalate serious client issues to Team leader in cases of untimely resolution
- Ensure all product information and disclosures are given to clients before and after the call/email requests
- Avoids legal challenges by monitoring compliance with service agreements
͏
- Handle technical escalations through effective diagnosis and troubleshooting of client queries
- Manage and resolve technical roadblocks/ escalations as per SLA and quality requirements
- If unable to resolve the issues, timely escalate the issues to TA & SES
- Provide product support and resolution to clients by performing a question diagnosis while guiding users through step-by-step solutions
- Troubleshoot all client queries in a user-friendly, courteous and professional manner
- Offer alternative solutions to clients (where appropriate) with the objective of retaining customers’ and clients’ business
- Organize ideas and effectively communicate oral messages appropriate to listeners and situations
- Follow up and make scheduled call backs to customers to record feedback and ensure compliance to contract SLA’s
͏
- Build people capability to ensure operational excellence and maintain superior customer service levels of the existing account/client
- Mentor and guide Production Specialists on improving technical knowledge
- Collate trainings to be conducted as triage to bridge the skill gaps identified through interviews with the Production Specialist
- Develop and conduct trainings (Triages) within products for production specialist as per target
- Inform client about the triages being conducted
- Undertake product trainings to stay current with product features, changes and updates
- Enroll in product specific and any other trainings per client requirements/recommendations
- Identify and document most common problems and recommend appropriate resolutions to the team
- Update job knowledge by participating in self learning opportunities and maintaining personal networks
͏
Deliver
No | Performance Parameter | Measure |
1 | Process | No. of cases resolved per day, compliance to process and quality standards, meeting process level SLAs, Pulse score, Customer feedback, NSAT/ ESAT |
2 | Team Management | Productivity, efficiency, absenteeism |
3 | Capability development | Triages completed, Technical Test performance |
Experience: 5-8 Years .
The expected compensation for this role ranges from $60,000 to $135,000 .
Final compensation will depend on various factors, including your geographical location, minimum wage obligations, skills, and relevant experience. Based on the position, the role is also eligible for Wipro's standard benefits including a full range of medical and dental benefits options, disability insurance, paid time off (inclusive of sick leave), other paid and unpaid leave options.
Applicants are advised that employment in some roles may be conditioned on successful completion of a post-offer drug screening, subject to applicable state law.
Wipro provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state, or local laws. Applications from veterans and people with disabilities are explicitly welcome.
Reinvent your world. We are building a modern Wipro. We are an end-to-end digital transformation partner with the boldest ambitions. To realize them, we need people inspired by reinvention. Of yourself, your career, and your skills. We want to see the constant evolution of our business and our industry. It has always been in our DNA - as the world around us changes, so do we. Join a business powered by purpose and a place that empowers you to design your own reinvention.