Senior Big Data Engineer (Java and Scala) (#16070777)

Our Client: Our client is an award-winning global tax services and software provider, is the largest Firm in the world dedicated exclusively to business taxes. With global headquarters in Dallas, Texas, the Firm provides an integrated suite of federal, state, local, and international tax services on a multijurisdictional basis, including tax recovery, consulting, advocacy, compliance, and technology services. It is a seven-time recipient of the International Service Excellence Award from the Customer Service Institute of America (CSIA) for its commitment to world-class client service.

Requirements:

• At least 4+ years of experience in software development using Java/Scala
• Experience with AWS
• Experience with Apache Spark 2.3, Spark SQL, Apache HBase or Cassandra
• Experience with old RDD API
• Be a team player in multi-national team
• Nice-to-have’s: Impala, Elasticsearch

Responsibilities:

The main goal for this role is to assist in migration of Java/Scala/Spark application from Cloudera on-premises deployment to AWS and modify existing application to work in new environment
It's important to deeply understand the current implementation of Compute Engine, written in Spark (Java), in order to:

• Profile the execution of the different work loads
• Make recommendations for improvements to the current codebase
• Implement short-term improvements if any identified
• Make recommendations for best practices in the use of the current codebase (i.e., “use this function rather than that other one”)
• Distill design guidelines for the next generation Compute Engine
• Additionally, as the team embarks on the migration to a cloud environment: Assist with the evaluation of the alternatives (EMR vs Cloudera CDP)
• Perform the code changes required to adapt the codebase to the new environment:
• Upgrade components