Trainee Data ATQA Engineer (#5867)

Our client is the Inflight Internet Company that is specialized in delivering reliable Internet connectivity and entertainment multimedia to aircrafts for the USA, enhancing the experience for both passengers and crew. It is the leading global provider of broadband connectivity products and services for aviation. It designs and source innovative network solutions that connect aircraft to the Internet, and develop software and platforms that enable customizable solutions for and by their aviation partners. Once connected, they provide industry leading reliability around the world. Their mission is to help aviation go farther by making planes fly smarter, so our aviation partners perform better and their passengers travel happier.

Responsibilities:

• Implementation of Data Quality Automation processors to verify data identity between databases and efficiently detect differences. This includes data research and program code development.
• Implementation of GreatExpectations Data Quality processors to check for unexpected data and special conditions. This involves both data research and program code development.
• Ensuring data integrity by checking for the absence of duplicates and other anomalies across different data sources.
• Validating the correctness of processors, ensuring their proper deployment using AWS, verifying the structure of GitHub repositories, and checking the availability of up-to-date documentation. This includes reviewing all aspects of the deployment to ensure everything was executed correctly.
• Creating defect tickets in case of any issues discovered during the process.
• Producing test results, compiling test reports, and documenting tickets with all necessary links and outcomes.
• Monitoring the operation of data quality processors, identifying problems as they arise, and finding solutions.

Qualifications:

• Bachelor’s degree or equivalent
• Strong data analysis skills with the capability to identify, analyze, and integrate various large complex data sources.
• AI usage experience
• A passion for working with huge data sets
• Proficiency in SQL and a solid understanding of RDBMS, given the diversity of data sources in use.
• Good Python skills, familiarity with the pandas tool is also a big plus.
• Experience with Bash commands and scripting.

Nice to have:

• Knowledge of Big Data technologies such as Hadoop, PySpark, Hive, and Airflow is advantageous