Middle QA Big Data (#4472)

N-iX is a global software development company founded in 2002, connecting over 2,400+ tech professionals across 40+ countries. We deliver innovative technology solutions in cloud computing, data analytics, AI, embedded software,IoT, and more to global industry leaders and Fortune 500 companies. Join us to create technology that drives real change for businesses and people across the world.

About the Client:

Our client is involved in a large-scale Data Transformation project, with a focus on solidifying the foundation of their data operations. They are aiming to ensure that data is accurate, consistent, and available at critical times to support their business needs. 

About the project:

The objective of this project is to enhance the QA processes through the implementation of automation within the Data landscape. This includes developing and maintaining regression testing suites, and creating test scripts and automation plans to improve the overall efficiency and quality of data testing practices.

Responsibilities:

• Design and execute data validation tests to ensure completeness in Azure Data Lake Storage (ADLS), Azure Synapse, and Databricks.
• Verify data ingestion, transformation, and loading (ETL/ELT) processes in Azure Data Factory (ADF).
• Validate data schema, constraints, and format consistency across different storage layers
• Conduct performance testing on data pipelines
• Optimize query performance by working with data engineers
• Identify, log, and track defects in JIRA.
• Collaborate with Data Engineers and Business Analysts to resolve data inconsistencies.
• Generate detailed test reports, dashboards, and documentation for stakeholders.

Requirements:

• 3+ years of QA experience with a strong focus on Big Data testing, particularly with hands-on experience in Data Lake environments on any cloud platform (preferably Azure).
• Experience with Azure.
• Hands-on experience in Azure Data Factory, Azure Synapse Analytics, or similar services. 
• Proficiency in SQL, capable of writing and optimizing both simple and complex queries for data validation and testing purposes.
• Experienced in PySpark, with experience in data manipulation and transformation, and a demonstrated ability to write and execute test scripts for data processing and validation (ability to understand the code and convert the logic to SQL).
• Hands-on experience with Functional & System Integration Testing in big data environments, ensuring seamless data flow and accuracy across multiple systems.
• Knowledge and ability to design and execute test cases in a behavior-driven development environment.
• Fluency in Agile methodologies, with active participation in Scrum ceremonies and a strong understanding of Agile principles.
• Familiarity with tools like Jira, including experience with X-Ray for defect management and test case management.
• Proven experience working on high-traffic and large-scale software products, ensuring data quality, reliability, and performance under demanding conditions.

Nice to Have:

• Familiarity with database administration and optimization techniques 
• Experience with Databricks.