For businesses around the world, data is one of their greatest assets. It is the oil of the modern world, the new most valuable resource. That’s why we are so passionate about our Big Data direction at N-iX.
So what is the best way to up your skills in Big Data? Work on a challenging project with an innovative tech stack? Learn from the best? Here’s what our Big Data engineers say about being part of the Big Data community at N-iX.
Igor
Delivery Manager
We started building our Big Data expertise only a few years ago. Today, we have a lot of Data projects at N-iX and a community of more than 60 engineers. One can even say that we’ve become a leader in this direction on the Ukrainian market 🙂
Together with our clients, we develop Data lakes, work with geospatial data, perform data migration to the cloud, real-time data streaming, ML solutions, and whatnot. We are doing projects ranging from aviation technology and telecom to industrial supply chain and retail. We work with Scala, Python, AWS, Azure, Spark, Hadoop, R, Kafka, Hive, and a lot of other technologies. This variety of tasks, projects, technologies, and domains allows our engineers to grow and always stay excited about their work. We partner with companies all around the world from the US and Europe to Japan and Jordan, which makes our projects even more diverse.
Rostyslav
Tech Lead (Gogo project)
I’ve been working on the Gogo project for over 3 years. Gogo is a leading provider of in-flight connectivity and wireless entertainment based in the US.
Gogo team at N-iX has around 20 people, most of whom work with data, whether it’s Big Data, Data Science or BI. We’ve got 2 main directions on the project. One is ingesting logs as we receive a colossal amount of logs from the planes – up to 5 terabytes a day. We parse this data for future analysis and processing. Also, we work on the development and maintenance of a Big Data platform on Amazon where we use EC2, S3, DynamoDB, and other AWS technologies.
The project is very dynamic, we have a lot of mini-projects (3-6 months), which make work much more interesting and challenging. For instance, one of the recent ones was analyzing the user experience during the first 15 minutes of the flight. What’s also great about this project is that we actively share knowledge in a team. So I’ve improved my knowledge of Scala, Hadoop, Spark, Finch, and AWS a lot.
I’m proud that we’ve successfully built a near real-time system for displaying the availability of the latency system when providing inflight Internet. Also, our client had an issue that satellite antennas on the planes needed to be changed very often in winter. And our Data Scientists helped identify what caused this problem. These antennas are very expensive, so it saved a lot of money for our client.
Andrian
Tech Lead (US telecom)
I’m doing Big Data for a large US telecom. For the first time in my career, I got an opportunity to start a project from scratch. Our team visited the client in Luxembourg, where we set the requirements and the project goals. This is a long-term project as it covers many phases. Our main task is developing a platform for the company’s business intelligence analysis. We started with building conceptual architecture, choosing the tech stack, and doing investigations. As it is a telecom company, the data covers calls, messages, charging information, and more. To develop a new platform, we are rebuilding the three old ones into the new one. We are working on the new functionality, the architecture, CI/CD pipelines, and all the development phases. Also, we do reverse engineering for the old system and migrate it to the new platform.
It is a real Big Data project. Our client is a big business so we process huge volumes of data. This project has given me a chance to do the full-cycle development of a Big Data application – from architecture to all development phases. Also, the client aims to reach exactly-once delivery semantic which requires much knowledge in distributed systems. And this is an interesting challenge.
Taras
Senior Data Engineer (financial and banking services company)
I like working on products that bring value to people. That’s why Telecom and Fintech are my favourite domains. Our client is a Jordan-based financial and banking services institution. Our team develops and supports data pipelines for banking processing. The main tasks include streaming data about transactions to the Data Warehouse and mobile applications as well as developing new functionality. We work with data from multiple databases that covers transactions, credit, debit, and Swift payments, applications for credits, customer care databases, electronic checks, and more. We mainly use Scala, Kafka, and AWS on this project.
Oles
Senior Big Data Engineer / Tech Lead (US industrial supply leader, Fortune 500)
We helped our client, a US Fortune 500 industrial supply company, start their Big Data direction. For me as a Data engineer, starting the project from the ground up is a great opportunity, especially when you work with a company that has more than 3 million customers.
Now, our main task is moving the existing customer solution to the cloud. At the current stage, we are building the infrastructure architecture on AWS. So we’ll know all its details and how to use it right when we start data processing. Also, we can test and choose the best technologies for the project. We tried and used multiple technologies, such as Kubernetes, Redshift, Snowflake, etc., did a lot of performance tests and benchmarking, and then decided to go with the best ones.
Igor
Senior Data Warehouse & BI engineer (US industrial supply leader, Fortune 500)
Our team is working on a project for the US-based supplier of maintenance, repair and operating products. It is a global company which has been on the market for almost 100 years. As a Big Data engineer, I find it interesting to work for such a huge company with a wide range of data. It never gets boring.
N-iX data engineers work in various customer’s teams. My team works with Financial data and implements requests from the company’s financial department. We help our client move data from different systems into Teradata Database. I enjoy working with Teradata. I believe this is one of the best databases due to its capacity and the possibilities it offers. But only if you use it right.
As our client is an industrial giant, the Data Warehouse operations are extremely important for them. We expect our team to grow and take care of the large part of the work in this field for the customer.
Colombia