Job Description
Role Purpose
The purpose of this role is to design, test and maintain software programs for operating systems or applications which needs to be deployed at a client end and ensure its meet 100% quality assurance parameters
͏
Do
-
Job Description: Airflow / Big Data Engineer
Role Summary
We are looking for a highly skilled Airflow / Big Data Engineer with strong hands-on experience in Apache Airflow, Python, and SQL. The ideal candidate will be responsible for building, orchestrating, and optimizing scalable data pipelines in a cloud-based big data environment.
Key Responsibilities
- Design, develop, and maintain robust Airflow DAGs to orchestrate complex ETL/ELT workflows with dependencies, retries, and monitoring.
- Develop Python-based data pipelines for data ingestion, transformation, validation, and automation.
- Write complex and optimized SQL queries for data extraction, transformation, and analysis.
- Integrate Airflow with cloud and big data services such as AWS Glue, EMR, Databricks, Snowflake, or similar platforms.
- Monitor, troubleshoot, and optimize data pipelines to ensure high performance, reliability, and scalability.
- Implement data quality checks, logging, alerting, and error handling mechanisms.
- Collaborate closely with data architects, analytics teams, and stakeholders to understand data requirements.
- Follow best practices for version control (Git), CI/CD, and documentation.
Mandatory Skills
- Strong expertise in Apache Airflow (DAG design, scheduling, monitoring, troubleshooting).
- Strong programming skills in Python for data engineering and automation.
- Strong SQL skills (complex joins, performance tuning, large datasets).
- Hands-on experience with Big Data / Cloud data platforms (AWS / Azure / GCP).
- Experience working with ETL/ELT pipelines and data warehousing concepts.
- Familiarity with Linux/Unix environments.
͏
2. Perform coding and ensure optimal software/ module development
- Determine operational feasibility by evaluating analysis, problem definition, requirements, software development and proposed software
- Develop and automate processes for software validation by setting up and designing test cases/scenarios/usage cases, and executing these cases
- Modifying software to fix errors, adapt it to new hardware, improve its performance, or upgrade interfaces.
- Analyzing information to recommend and plan the installation of new systems or modifications of an existing system
- Ensuring that code is error free or has no bugs and test failure
- Preparing reports on programming project specifications, activities and status
- Ensure all the codes are raised as per the norm defined for project / program / account with clear description and replication patterns
- Compile timely, comprehensive and accurate documentation and reports as requested
- Coordinating with the team on daily project status and progress and documenting it
- Providing feedback on usability and serviceability, trace the result to quality risk and report it to concerned stakeholders
͏
3. Status Reporting and Customer Focus on an ongoing basis with respect to project and its execution
- Capturing all the requirements and clarifications from the client for better quality work
- Taking feedback on the regular basis to ensure smooth and on time delivery
- Participating in continuing education and training to remain current on best practices, learn new programming languages, and better assist other team members.
- Consulting with engineering staff to evaluate software-hardware interfaces and develop specifications and performance requirements
- Document and demonstrate solutions by developing documentation, flowcharts, layouts, diagrams, charts, code comments and clear code
- Documenting very necessary details and reports in a formal way for proper understanding of software from client proposal to implementation
- Ensure good quality of interaction with customer w.r.t. e-mail content, fault report tracking, voice calls, business etiquette etc
- Timely Response to customer requests and no instances of complaints either internally or externally
͏
Deliver
| No. | Performance Parameter | Measure |
| 1. | Continuous Integration, Deployment & Monitoring of Software | 100% error free on boarding & implementation, throughput %, Adherence to the schedule/ release plan |
| 2. | Quality & CSAT | On-Time Delivery, Manage software, Troubleshoot queries, Customer experience, completion of assigned certifications for skill upgradation |
| 3. | MIS & Reporting | 100% on time MIS & report generation |
Experience: 5-8 Years .
Reinvent your world. We are building a modern Wipro. We are an end-to-end digital transformation partner with the boldest ambitions. To realize them, we need people inspired by reinvention. Of yourself, your career, and your skills. We want to see the constant evolution of our business and our industry. It has always been in our DNA - as the world around us changes, so do we. Join a business powered by purpose and a place that empowers you to design your own reinvention.