Data Engineer (AI-ML Data Systems)

$110K - $190K Reston, VA, US Mid Level Data Engineer

Interested in this Data Engineer role at Pantheon Data?

Apply Now →

Skills & Technologies

AwsDockerEmbeddingsPgvectorPythonRagVector Search

About This Role

AI job market dashboard showing open roles by category

Company Overview

Pantheon Data (a Kenific Holding company) is a private, small business based in the Washington, DC, area. Pantheon Data was founded in 2011, initially providing acquisition and supply chain management services to the US Coast Guard. Our service offerings have grown in the past ten years, including infrastructure resiliency, contact center operations, information technology, software engineering, program management, strategic communications, engineering, and cybersecurity. We have also grown our customer base to include commercial clients. The company has used this experience to expand our service offerings to other agencies within the Department of Homeland Security (DHS), the Department of Defense (DoD), and other Federal Civilian Agencies.

Position Overview

We are seeking a hands-on Data Engineer to help design, build, and operate the data foundations that support advanced analytics, AI/ML, and intelligent document processing solutions. The right candidate is a strong engineer who understands how data moves through real systems: ingestion, orchestration, transformation, quality checks, storage, query patterns, operational monitoring, and delivery to downstream applications. This person should be comfortable working across structured, semi-structured, and unstructured data held in relational, graph, and vector data stores, and should bring the judgment to build pipelines that are reliable, explainable, maintainable, and useful to the engineering teams and products that depend on them.

The ideal candidate has strong Python and SQL skills, understands when data should be modeled for operational use versus analytical use, and can reason clearly about batch processing, event-driven pipelines, data quality, lineage, and downstream consumption. They should be able to become productive quickly in a complex engineering environment, ask good questions, and build systems that other engineers can trust and extend. Experience with AWS, graph databases and query languages such as AQL, vector search, document data, or AI/ML data pipelines is valuable, but the core requirement is strong data engineering judgment: knowing how to move, structure, validate, and serve data reliably in support of real products and mission needs.

This role owns the data pipelines and data stores. You will partner closely with the Machine Learning Engineers, who own models, retrieval, extraction, and evaluation: they define how documents are chunked, embedded, enriched, and retrieved, and you build and run the pipelines and stores that apply those designs reliably at scale.

Responsibilities

  • Design, build, and maintain reliable data pipelines for structured, semi-structured, and unstructured data sources.
  • Develop Python-based processing workflows for data ingestion, normalization, validation, enrichment, and transformation.
  • Work with SQL and relational data stores to support transactional, analytical, and application-facing use cases.
  • Work with graph databases and query languages such as ArangoDB/AQL to model relationships between entities, documents, and metadata, and to support traversal and relationship-driven queries.
  • Build and operate vector data pipelines and stores: run embedding generation at scale; manage vector index builds, refreshes, backfills, and re-embedding when models change; and serve the lookups that retrieval and RAG workloads depend on, implementing the chunking, embedding, and metadata specifications defined by the Machine Learning Engineers.
  • Help design data models and storage patterns appropriate to the workload, including OLTP, OLAP, object storage, document-oriented, search, vector, or graph-oriented patterns when applicable.
  • Implement orchestration and scheduling for repeatable data workflows using tools such as Airflow, AWS Step Functions, Dagster, Prefect, Glue workflows, or similar technologies.
  • Build automated quality checks, reconciliation logic, validation reports, and operational alerts so data issues are detected early and can be diagnosed quickly.
  • Support data pipelines that feed AI/ML, retrieval, document intelligence, analytics, and application workflows.
  • Collaborate with machine learning engineers, software engineers, cloud engineers, and product stakeholders to turn ambiguous data problems into working software.
  • Write maintainable code, participate in code reviews, document data flows, and contribute to engineering standards for testing, deployment, observability, and version control.
  • Help improve the velocity of a growing engineering team by taking ownership of well-scoped data engineering work while continuing to grow into broader system ownership.

Required Skills and Experience

  • Bachelor's degree in Computer Science, Engineering, or a related technical field from an accredited university.
  • 5+ years of professional hands-on data engineering, software engineering, analytics engineering, or closely related experience.
  • Strong Python programming skills, including experience writing maintainable production-oriented code rather than only notebooks or one-off scripts.
  • Strong SQL skills and practical understanding of data modeling, query performance, joins, indexing, schemas, normalization/denormalization, and data quality.
  • Understanding of core data engineering concepts, including batch processing, event-driven workflows, ETL/ELT, orchestration, idempotency, retries, backfills, lineage, and failure handling.
  • Working knowledge of OLTP versus OLAP systems and the tradeoffs between transactional databases, analytical stores, object storage, search-oriented systems, and graph and vector databases.
  • Experience building or supporting data pipelines that move data between systems, such as APIs, databases, files, object storage, queues, warehouses, or downstream applications.
  • Ability to reason about data correctness, schema changes, validation, reconciliation, duplicate handling, missing data, and operational recovery.
  • Comfortable working with Git, pull requests, code review, issue tracking, documentation, and collaborative software development practices.
  • Strong communication skills and the ability to explain data flow, design choices, limitations, and tradeoffs to both technical and non-technical stakeholders.
  • Ability to work effectively in a distributed, cross-functional engineering environment and produce high-quality work with limited hand-holding.
  • Ability to work effectively remotely in cross-functional teams.
  • Ability to meet deadlines and produce quality work.
  • Proficient in Microsoft Suite software including Outlook, Word, Excel, SharePoint, and PowerPoint.

Preferred Skills and Experience

  • AWS data services such as S3, Lambda, Glue, Athena, Step Functions, SQS/SNS, Kinesis, EMR, RDS, DynamoDB, Redshift, OpenSearch, or CloudWatch.
  • Experience with workflow orchestration tools such as Airflow, Dagster, Prefect, AWS Step Functions, Glue, or similar.
  • Experience with PySpark, Spark, Databricks, EMR, Snowflake, Redshift, or other distributed/analytical data platforms.
  • Experience supporting AI/ML or RAG-style data workflows, including metadata enrichment, retrieval datasets, vector search, embeddings, evaluation datasets, or human validation workflows.
  • Experience with document-oriented or unstructured data pipelines, including PDFs, OCR output, tables, forms, images, extracted text, metadata, or search indexes.
  • Hands-on experience with graph databases and query languages such as ArangoDB/AQL, Neo4j/Cypher, or Gremlin, including graph-shaped data modeling.
  • Experience with vector databases or vector search engines such as pgvector, OpenSearch, or similar, including index tuning and hybrid (vector + lexical) lookups.
  • Experience with Docker, CI/CD, infrastructure as code, automated testing, logging, monitoring, and production support is a plus.
  • Familiarity with data governance, access control, PII handling, auditability, lineage, and compliance-sensitive environments.

Clearance Requirements

Applicants selected will be subject to a security investigation and may need to meet eligibility requirements. Candidates must have the ability to obtain and maintain a Secret clearance; an active Secret clearance is preferred.

Work Location: Reston, VA - Remote

  • Our company prioritizes the benefits of flexibility and collaboration, whether that happens in person or remotely.
  • If the position is remote or hybrid, you may periodically work from a Pantheon Data office location or client site.
  • If this position is assigned to a Pantheon Data office location or client site, you'll work with colleagues and clients in person, as needed for specific client requirements.

Interview Requirement: Candidates who are local to the area should be prepared to participate in an in-person interview as part of the selection process. Candidates outside the local area may be considered for a virtual interview.

Compensation

The salary range for this position is $110,000 - $190,000. This is not, however, a guarantee of compensation or salary. Rather, salary will be set based on experience, geographic location and possibly contractual requirements and could fall outside of this range.

Benefits Overview

We are always looking for good people! Pantheon Data is committed to providing its employees with competitive salaries and benefits in order to increase employee satisfaction and productivity.In addition to our benefits, we also offer SmartBenefits through the Washington Metro Area Transportation Authority, where you specify an amount of your pre-tax wages be paid directly to your SmarTrip account. In some cases, tuition assistance may be available for continuing education expenses and certifications related to their position. Additional details may be found at https://pantheon-data.com/careers/

Pantheon Data Important Information

All qualified applicants will be considered for employment without regard to disability, status as a protected veteran, or any other status protected by applicable federal, state, local, or international law.

As part of the application process, you are expected to be on camera during interviews and assessments. We reserve the right to take your picture to verify your identity and prevent fraud.

If you require reasonable accommodation in completing this application, interviewing, completing any pre-employment testing, or otherwise participating in the employee selection process, please direct your inquiries to our Talent Team at Recruiting@pantheon-data.com or by phone (571) 363-4020.

This company uses E-Verify to confirm each employee's work authorization. For more information, click here E-Verify Participation Poster

Salary Context

This $110K-$190K range is below the median for Data Engineer roles in our dataset (median: $155K across 31 roles with salary data).

Role Details

Company Pantheon Data
Title Data Engineer (AI-ML Data Systems)
Location Reston, VA, US
Category Data Engineer
Experience Mid Level
Salary $110K - $190K
Remote No

About This Role

Data Engineers build the pipelines that feed AI models. They design ETL workflows, manage data lakes, and ensure training and inference data is clean, timely, and accessible. Without good data engineering, AI projects fail. It's that simple.

The AI era has expanded the data engineer's scope far beyond batch ETL jobs. You're building real-time embedding pipelines for RAG systems, managing vector databases, ensuring training data quality at scale, and building the infrastructure that lets ML teams iterate on data as fast as they iterate on models. Data quality is the biggest predictor of model quality, and you're the person responsible for it.

Across the 10,419 AI roles we're tracking, Data Engineer positions make up 1% of the market. At Pantheon Data, this role fits into their broader AI and engineering organization.

Data Engineer demand in AI contexts is strong and growing. Every company building AI needs clean, reliable data pipelines. The shift toward real-time AI applications (chatbots, recommendation engines, agent systems) means data engineering is more critical than ever. Companies are willing to pay premium salaries for data engineers with AI/ML pipeline experience.

What the Work Looks Like

A typical week includes: debugging a data pipeline that's producing stale embeddings for the RAG system, optimizing a Spark job that processes training data, building a data quality monitoring dashboard, meeting with the ML team to understand their next data requirements, and writing dbt models that transform raw event data into ML-ready features. The work is deeply technical and high-impact.

Data Engineer demand in AI contexts is strong and growing. Every company building AI needs clean, reliable data pipelines. The shift toward real-time AI applications (chatbots, recommendation engines, agent systems) means data engineering is more critical than ever. Companies are willing to pay premium salaries for data engineers with AI/ML pipeline experience.

Skills Required

Aws (11% of roles) Docker (4% of roles) Embeddings (3% of roles) Pgvector (1% of roles) Python (23% of roles) Rag (9% of roles) Vector Search (2% of roles)

SQL, Python, and distributed systems (Spark, Airflow, dbt) are core. Cloud data platforms (Snowflake, BigQuery, Redshift) are increasingly standard. Many AI-focused roles also want familiarity with vector databases and embedding pipelines. Understanding data modeling, pipeline orchestration, and data quality frameworks covers the essentials.

AI-specific data engineering skills include: building feature stores, managing training data versioning, implementing data lineage tracking, and building real-time embedding pipelines. Experience with streaming systems (Kafka, Flink) is valuable for real-time AI applications. Understanding ML data requirements (balanced datasets, data augmentation, evaluation set construction) makes you much more effective working with ML teams.

Strong postings specify the data stack, mention ML pipeline work, and describe the scale of data you'll be working with. Look for companies that understand the connection between data quality and model quality. Avoid roles that conflate data engineering with data analysis.

Compensation Benchmarks

Data Engineer roles pay a median of $178,650 based on 241 positions with disclosed compensation. Mid-level AI roles across all categories have a median of $193,200. This role's midpoint ($150K) sits 16% below the category median. Disclosed range: $110K to $190K.

Across all AI roles, the market median is $214,834. Top-quartile compensation starts at $266,000. The 90th percentile reaches $321,730. For comparison, the highest-paying categories include AI Safety ($274,500) and Research Engineer ($252,000). By seniority level: Entry: $114,400; Mid: $193,200; Senior: $230,000; Director: $272,100; VP: $235,000.

Pantheon Data AI Hiring

Pantheon Data has 2 open AI roles right now. They're hiring across AI/ML Engineer, Data Engineer. Based in Reston, VA, US. Compensation range: $190K - $220K.

Location Context

Across all AI roles, 6% (580 positions) offer remote work, while 9,776 require on-site attendance. Top AI hiring metros: New York (3,785 roles, $222,480 median); San Francisco (3,262 roles, $261,300 median); Los Angeles (1,896 roles, $210,000 median).

Career Path

Common paths into Data Engineer roles include Backend Engineer, Database Administrator, Analytics Engineer.

From here, career progression typically leads toward Senior Data Engineer, ML Engineer, Data Platform Lead.

Master SQL and Python first. Then learn a distributed processing framework (Spark or its modern alternatives) and a pipeline orchestrator (Airflow, Dagster, Prefect). Build a portfolio project that demonstrates end-to-end pipeline construction: ingest, transform, validate, serve. If you want to specialize in AI data engineering, add vector databases and embedding pipelines to your skill set.

What to Expect in Interviews

Expect SQL deep-dives (query optimization, partitioning strategies, data modeling), Python coding focused on data pipeline patterns, and system design questions about building scalable ETL workflows. Companies with ML teams will ask about feature stores, embedding pipelines, and training data management. Be ready to discuss data quality monitoring, pipeline orchestration, and how you'd handle schema evolution in a production data lake.

When evaluating opportunities: Strong postings specify the data stack, mention ML pipeline work, and describe the scale of data you'll be working with. Look for companies that understand the connection between data quality and model quality. Avoid roles that conflate data engineering with data analysis.

AI Hiring Overview

The AI job market has 10,419 open positions tracked in our dataset. By seniority: 117 entry-level, 4,975 mid-level, 3,840 senior, and 1,487 leadership roles (Director, VP, C-Level). Remote roles make up 6% of the market (580 positions). The remaining 9,776 roles require on-site or hybrid attendance.

The market median for AI roles is $214,834. Top-quartile compensation starts at $266,000. The 90th percentile reaches $321,730. Highest-paying categories: AI Safety ($274,500 median, 90 roles); Research Engineer ($252,000 median, 580 roles); AI Engineering Manager ($250,811 median, 72 roles).

Data Engineer demand in AI contexts is strong and growing. Every company building AI needs clean, reliable data pipelines. The shift toward real-time AI applications (chatbots, recommendation engines, agent systems) means data engineering is more critical than ever. Companies are willing to pay premium salaries for data engineers with AI/ML pipeline experience.

The AI Job Market Today

The AI job market spans 10,419 open positions across 16 role categories. The largest categories by volume: AI/ML Engineer (7,849), AI Product Manager (516), AI Software Engineer (513). These three account for the majority of open positions, though smaller categories often have higher per-role compensation because of specialized skill requirements.

The seniority mix tells a story about where AI teams are in their maturity. Entry-level roles (117) are outnumbered by mid-level (4,975) and senior (3,840) positions, reflecting that most companies are past the 'build a team from scratch' phase and need experienced engineers who can ship production systems. Leadership roles (Director, VP, C-Level) total 1,487 positions, representing the bottleneck between technical execution and organizational strategy.

Remote work availability sits at 6% of all AI roles (580 positions), with 9,776 requiring on-site or hybrid attendance. The remote share has stabilized after the post-pandemic correction. Senior and specialized roles (Research Scientist, ML Architect) are more likely to be remote-eligible than entry-level positions, partly because experienced hires have more negotiating power and partly because these roles require less hands-on mentorship.

AI compensation is structured in clear tiers. The market median sits at $214,834. Top-quartile roles start at $266,000, and the 90th percentile reaches $321,730. These figures include base salary with disclosed compensation. Total compensation (including equity, bonuses, and sign-on) runs 20-40% higher at companies that offer those components.

Category matters for compensation. AI Safety roles lead at $274,500 median, while Prompt Engineer roles sit at $140,000. The spread between highest and lowest-paying categories reflects the premium on specialized technical skills versus broader analytical roles.

The most in-demand skills across all AI postings: Python (2,350 postings), Aws (1,173 postings), Azure (1,030 postings), Rag (986 postings), Gcp (714 postings), Pytorch (649 postings), Kubernetes (583 postings), Claude (581 postings). Python dominates, appearing in the vast majority of role descriptions regardless of category. Cloud platform experience (AWS, GCP, Azure) is the second most common requirement. The newer entrants to the top skills list (RAG, vector databases, LLM APIs) reflect the shift from traditional ML toward generative AI applications.

Frequently Asked Questions

Based on 241 roles with disclosed compensation, the median salary for Data Engineer positions is $178,650. Actual compensation varies by seniority, location, and company stage.
SQL, Python, and distributed systems (Spark, Airflow, dbt) are core. Cloud data platforms (Snowflake, BigQuery, Redshift) are increasingly standard. Many AI-focused roles also want familiarity with vector databases and embedding pipelines. Understanding data modeling, pipeline orchestration, and data quality frameworks covers the essentials.
About 6% of the 10,419 AI roles we track offer remote work. Remote availability varies by company and seniority level, with senior and leadership roles more likely to offer location flexibility.
Pantheon Data is among the companies actively hiring for AI and ML talent. Check our company profiles for detailed breakdowns of open roles, salary ranges, and hiring trends.
Common next steps from Data Engineer positions include Senior Data Engineer, ML Engineer, Data Platform Lead. Progression depends on whether you lean toward technical depth, people management, or product strategy.

Get Weekly AI Career Intelligence

Salary data, skills demand, and market signals from 16,000+ AI job postings. Every Monday.