Azure Data Engineer Job at Vivsoft Technologies, Washington DC

RXdwZ0k4VmJoM0Z3emJwVURERWV5SXEvcXc9PQ==
  • Vivsoft Technologies
  • Washington DC

Job Description


About the Company
VivSoft is an emerging technology company specializing in leveraging modern technologies to solve our clients’ most complex mission challenges. We focus on Cloud, Enterprise DevSecOps, Artificial Intelligence (AI), and Digital Customer Experience to drive mission-enabling digital transformation.
Our passion lies in building mission-focused, open, and scalable solutions. We are a diverse team of strategists, engineers, designers, and innovators with deep experience in developing high-performance software and AI factory accelerators through automation and agile practices.

Position Overview:
We are seeking a highly skilled Azure Databricks Engineer to design, build, and optimize large-scale data pipelines and lakehouse solutions within the Azure ecosystem. The ideal candidate will have deep technical expertise in Apache Spark, Delta Lake, and Azure Data Services, along with a strong understanding of data modeling, ETL/ELT processes, and performance optimization.

Key Responsibilities:
  • Design, build, and maintain ETL/ELT pipelines using Azure Databricks, Delta Lake, and Apache Spark for structured and unstructured data.
  • Implement data ingestion and transformation processes integrating multiple internal and external data sources.
  • Architect and optimize data lakehouse solutions, ensuring efficient data storage, retrieval, and partitioning strategies.
  • Develop and manage batch and real-time streaming data pipelines using Structured Streaming and other Azure services.
  • Configure and manage Databricks workspaces, clusters, and compute resources for optimal performance and cost efficiency.
  • Implement data security, access control, and governance measures within the Azure environment.
  • Collaborate with data scientists to deploy and operationalize machine learning models.
  • Utilize MLflow for model tracking, versioning, and automated deployment pipelines.
  • Monitor pipeline performance, troubleshoot issues, and optimize Spark jobs for improved efficiency and reduced processing costs.
  • Implement data quality checks and ensure reliability across all data processing workflows.
  • Work closely with stakeholders to translate business needs into scalable data solutions.
  • Demonstrate strong knowledge of ETL/ELT frameworks, data modeling, and big data technologies.
  • Hands-on experience with Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), and Azure Synapse Analytics.
  • Proficient in Apache Spark, Delta Lake, Python, SQL, and Scala.
  • Minimum 5+ years of experience as a Data Engineer, with 3+ years specifically in Azure Databricks and related Azure services.
Skills Required:
  • Bachelor’s or Master’s degree in Computer Science, Engineering, Data Science, or a related field.
  • Strong understanding of data architecture, performance tuning, and cost optimization in Azure.
  • Excellent analytical, problem-solving, and communication skills.
  • Experience working in Agile environments and implementing data governance frameworks.
  • Preferred certifications include Microsoft Certified: Azure Data Engineer Associate (DP-203) or Databricks Certified Data Engineer.
  • Passion for innovation, continuous improvement, and driving efficiency in large-scale data environments.
  • Opportunity to work with cutting-edge Azure data and AI technologies within a collaborative, high-impact environment.

Job Tags

Full time,

Similar Jobs

Headway Workforce Solutions

Part-Time Field Interviewer Job at Headway Workforce Solutions

 ...Job Description Job Description Field Interviewer Part-Time, Temporary (AprilSeptember 2025) Location : Minnesota (Various Facilities) Pay: $23.50/hour + mileage & drive-time reimbursement Are you looking for a meaningful opportunity to make an impact... 

Lotus Sales

Paid Summer Sales Internship - No Experience + Training Job at Lotus Sales

 ...Job Description Job Description Summer Sales Internship Earn Big, Learn Fast, Grow Faster Spend your summer developing real sales skills, working with a driven team, and earning performance-based income. Housing is available for students who relocate. Highlights... 

Elevo Learning

Student Wellness Coach Job at Elevo Learning

Overview Join to apply for the Student Wellness Coach role at Elevo Learning .About Us: Elevos mission is to transform the well-being of students so they look forward to coming to school. Our WASC-accredited curriculum engages students and fosters social-emotional learning... 

ATD Technology LLC

Pharmacist Job at ATD Technology LLC

 ...Skills: One (1) year of Pharmacist experience. Pharmacy dispensing system experience. Sterile Processing experience. IV room experience. Knowledge of an EMR (Electronic Medical Records system) preferred. Spanish preferred Education... 

The TEA Center

3 Days on Weekdays/ Weekend Administrative Assistant for a Childcare Center Job at The TEA Center

 ...be part of a growing business while making a difference in the lives of children. Prior experience in a childcare setting a plus on weekdays but a MUST for weekend admin. The program values great leadership and opportunity for growthDay to day operations Operations....