Job Description
Role: Data Engineer
Primary Skills: Azure Databricks, Azure Data Factory, PySpark, SQL, Azure Data Lake
Location: Mount Laurel, NJ (3 Days onsite/week)
Role Overview
We are looking for a skilled Data Engineer with strong hands-on experience in Azure Databricks and Azure Data Factory to design, develop, and maintain scalable data pipelines and cloud-based data solutions. The ideal candidate will work closely with data architects, analysts, business stakeholders, and application teams to build reliable ETL/ELT workflows, transform large datasets, and enable analytics and reporting use cases.
Key Responsibilities
• Design, develop, and maintain scalable data pipelines using Azure Data Factory and Azure Databricks.
• Build ETL/ELT workflows for data ingestion, transformation, cleansing, enrichment, and loading into data lakes or data warehouses.
• Develop and optimize PySpark, Spark SQL, and SQL scripts for large-scale data processing.
• Integrate data from multiple sources such as databases, APIs, flat files, cloud storage, and enterprise applications.
• Implement incremental data loading, scheduling, parameterization, and reusable pipeline frameworks in ADF.
• Work with Azure Data Lake Storage, Delta Lake, and lakehouse architecture patterns for structured and unstructured data.
• Monitor, troubleshoot, and optimize data pipelines for performance, reliability, cost efficiency, and data quality.
• Collaborate with business users, data analysts, data scientists, and architects to understand requirements and deliver data solutions.
• Implement data validation, reconciliation, exception handling, logging, and alerting mechanisms.
• Follow coding standards, version control, CI/CD, and deployment best practices using tools such as Git and Azure DevOps.
• Ensure data security, access control, and compliance by applying Azure security best practices.
• Prepare technical documentation including data flow diagrams, mapping documents, pipeline design, and support guides.
Required Technical Skills
• Strong hands-on experience with Azure Databricks, Databricks notebooks, clusters, jobs, and workflows.
• Strong experience in Azure Data Factory including pipelines, datasets, linked services, triggers, parameters, variables, and integration runtimes.
• Good programming experience in PySpark, Python, Spark SQL, and SQL.
• Experience working with Azure Data Lake Storage Gen2, Delta Lake, and data lakehouse concepts.
• Knowledge of ETL/ELT design patterns, data warehousing concepts, dimensional modeling, and data integration methods.
• Experience in performance tuning of Spark jobs, SQL queries, and ADF pipelines.
• Understanding of batch processing, incremental loading, CDC concepts, and data partitioning strategies.
• Experience with Git, Azure DevOps, CI/CD pipelines, and release management for data engineering solutions.
• Knowledge of data quality checks, monitoring, logging, and error handling frameworks.
• Basic understanding of Azure security concepts such as managed identities, service principals, Key Vault, RBAC, and private endpoints.
Preferred / Good-to-Have Skills
• Experience with Azure Synapse Analytics, Azure SQL Database, SQL Server, or Snowflake.
• Knowledge of Unity Catalog, data governance, metadata management, and data lineage.
• Experience with streaming data processing using Kafka, Event Hubs, or Databricks Structured Streaming.
• Exposure to Power BI, reporting platforms, or analytics consumption layers.
• Experience working in Agile delivery models and cross-functional project teams.
• Microsoft Azure Data Engineer certification or Azure Databricks certification is an added advantage.
Experience: 5-8 Years .
The expected compensation for this role ranges from $60,000 to $135,000 .
Final compensation will depend on various factors, including your geographical location, minimum wage obligations, skills, and relevant experience. Based on the position, the role is also eligible for Wipro's standard benefits including a full range of medical and dental benefits options, disability insurance, paid time off (inclusive of sick leave), other paid and unpaid leave options.
Applicants are advised that employment in some roles may be conditioned on successful completion of a post-offer drug screening, subject to applicable state law.
Wipro provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state, or local laws. Applications from veterans and people with disabilities are explicitly welcome.
Reinvent your world. We are building a modern Wipro. We are an end-to-end digital transformation partner with the boldest ambitions. To realize them, we need people inspired by reinvention. Of yourself, your career, and your skills. We want to see the constant evolution of our business and our industry. It has always been in our DNA - as the world around us changes, so do we. Join a business powered by purpose and a place that empowers you to design your own reinvention.