Data Engineer Job at Stellar IT Solutions LLC, Hopkins, MN

MGlaVmMrUWdSN2lzZlFSY01IVmpSaGNJVlE9PQ==
  • Stellar IT Solutions LLC
  • Hopkins, MN

Job Description

Job Title: Data Engineer

Interview: Virtual

Job Duration: 6 Months

Job Location: Hopkins MN



Job Description:
We are seeking a Data Engineer with expertise in building scalable data pipelines and managing big data solutions in Azure. This role involves collaborating with crossfunctional teams to design develop and optimize data workflows while ensuring alignment with industry standards and best practices.

Key Responsibilities:

  • Design and develop data pipelines in Azure using Spark technologies to process and manage large datasets.
  • Build and maintain Zafin and AC360 data pipelines to ensure seamless integration and highperformance data processing.
  • Develop Airflow DAGs for automated scheduling and orchestration of workflows.
  • Collaborate with product owners business stakeholders and engineering teams to gather and define technical requirements.
  • Apply a DevOps mindset to production systems incorporating automation active alerting and selfhealing techniques.
  • Optimize big data solutions to ensure scalability reliability and efficiency .
  • Perform peer code reviews and define best engineering practices to maintain highquality deliverables.
  • Document technical designs workflows and engineering artifacts to support team knowledge sharing and maintenance.
  • Partner with key stakeholders to ensure endtoend testing and smooth deployment of software products.

Qualifications:

  • Big Data Expertise : 10 years of experience managing largescale datasets and distributed systems.
  • Spark (Scala SQL Py) : 5 years of handson experience.
  • Cloud Technologies : 4 years of experience with Azure Kubernetes and PostGres.
  • DevOps and CI/CD : Proficiency in Maven Git Jenkins Docker and Kubernetes.
  • Programming Skills : Strong knowledge of Python Java and GoLang (3 years).
  • Airflow and Elastic Stack : 3 years of experience with tools like Kibana and ELK.
  • Soft Skills : Excellent verbal and written communication skills (10 years).
  • Analytical and ProblemSolving Skills : Strong technical and logical capabilities (10 years).

Preferred Skills:

  • Experience with Spark Scala Python and the Azure technology stack.
  • Familiarity with microservicesbased applications and best practices for data engineering.

Job Tags

Remote job,

Similar Jobs

Reveation Labs

$15 - $20 per hour - Hiring Now - Remote Data Entry Operator (Full-Time) Job at Reveation Labs

 ...months of experience in Data Entry/Admin Must be proficient in MS Office including Excel (Formulas) Must have good analytical skills and attention to detail Must have excellent typing speed over 50 WPM Employment Type: Full-Time Salary: $ 15.00 20.00 Per Hour

Papa John's

Delivery Driver - RUSH DRIVERS - White Oak Job at Papa John's

Receive the following: Hourly Wage - count on a regular paycheck Mileage paid per mile driven - Paid Daily Food Discounts Health Benefits Promotion opportunities Locally owned Papa John's Pizza locations seeking Drivers who like to take home cash daily...

System One

Technical Recruiter Job at System One

 ...Technical Recruiter Rockville, MD (Hybrid) Are you an experienced Technical Recruiter looking for a new challenge? ALTA is...  ...from you. Join our team of talented recruiters and help us build a world-class technology consulting firm. Apply today! #M2... 

California Correctional Institution

CASE RECORDS TECHNICIAN Job at California Correctional Institution

 ...eligible for a training and Development Assignment may also be considered for this position (s). * This advertisement is for a Case Records Technicianpositionlocated at the California Correctional Institution in Tehachapi, CA Kern County. Please indicate the position... 

Closets by Design Indianapolis

Sales/Designer - Fun & Creative Job at Closets by Design Indianapolis

 ...can help others organize their home. As a Closets by Design...  ...can enjoy the flexibility of working from your home and our clients...  ...receive complete sales and design training, so no experience is necessary...  ...Monthly Bonus opportunity* Paid training* A flexible schedule...