Job#: 2060128

Job Description:

Data Engineer X3 Level (4 years of experience)

Competition: reaching out to at least one other company

Location: Bentonville, AR (remote on contract- must relocate for perm)

Projects: Finance/ Sourcing/ Luminate (data ventures)

Rate: TBD (can’t use $104 Data Scientist) (aim for at least $92)

Total: 4 CTH

Connors notes:

Programming languages: Python (most important), Scala/ Spark

Cloud: GCP- required

Message Q: Kafka- required

Databases: Big Query, SQL, Hive, Map Reduce

Sponsorship: Can sponsor but does not want to wait for i 140. Must be able to convert right at 3 months

YoE: 4 years of experience

Interview process: 30-60 minute conversation with manager, no coding.

Manager provided JD:

• Designing and building ETL pipeline using Sqoop, Hive, Map Reduce and Spark on on-prem and cloud environments;
• Functional programming using Python and Scala for complex data transformations and in-memory computations;
• Uusing Erwin for Logical/Physical data modeling and Dimensional Data Modeling;
• Designing and developing UNIX/Linux scripts for handing complex file formats and structures;
• Orchestration of workflows and jobs using Airflow and Automic;
• Creating multiple Kafka producers and consumers for data transferring;
• Performing Continuous Integration and deployment (CI/CD) using tools like Git, Jenkin to run test cases and build applications with code coverage using Scala test;
• Analyzing data using SQL, Big Query
• Monitoring the cluster performance, setting up alerts, documenting the designs, workflow.
• Providing production support, troubleshooting, and fixing the issues by tracking the status of running applications to perform System administrator tasks. need good communication