Interested in this AI/ML Engineer role at Navy Federal Credit Union?
Apply Now →Skills & Technologies
About This Role
*Navy Federal Credit Union currently does not provide sponsorship for this role. Applicants must be authorized to work in the United States without the need for current or future sponsorship.*
The Cognitive and Generative AI Engineering team is responsible for developing and implementing AI\-driven solutions that enhance and scale AI adoption across Navy Federal Credit Union. The Principal AI Engineer is a senior, hands\-on technical leader responsible for end\-to\-end delivery of production\-grade AI capabilities that span architecture, implementation, deployment, and operational readiness while translating complex technical work into clear, executive\-ready narratives. This is a pivotal role in the AI Center for Enablement, leading the design and implementation of cutting\-edge AI systems. This role leverages and adapts state\-of\-the\-art large language models (LLMs) to solve complex business problems and identifies opportunities to build modular and reusable components. The incumbent will collaborate with ETS (Enterprise Technology Services) and Business partners including Enterprise Architecture, Enterprise AI Strategy, and the AI Working Group to build and drive solutions. They will provide delivery and ongoing support for NFCU's data science, advanced analytics, and augmented intelligence technologies, executing on the strategic vision and ensuring the successful implementation of AI initiatives across the organization. Successful candidates will exhibit excellent problem\-solving skills, effective communication and analytical skills, as well as strong leadership qualities with a proven track record of delivering measurable outcomes.
Responsibilities
- Lead delivery of AI solutions from discovery through production, ensuring predictable execution, risk management, and measurable value realization.
- Define and socialize technical delivery plans, milestones, dependencies, and release readiness criteria; proactively surface tradeoffs and decision points.
- Translate business problems into implementable designs and backlog\-ready work, maintaining alignment across engineering, security/risk, architecture, and business stakeholders.
- Architect and implement scalable AI systems, including data pipelines, model training/inference patterns, and integration into applications/services.
- Build and deploy AI solutions leveraging modern AI platforms and orchestration patterns (e.g., LLM/agent workflows, RAG, model grounding, evaluation gates, safety controls) as applicable to the use case.
- Establish and enforce strong engineering discipline: code quality, automated testing, performance tuning, observability, and operational runbooks for AI services.
- Lead by example in debugging complex issues, completing critical path implementation work, and unblocking teams through direct technical contribution.
- Develop concise, executive\-ready narratives (1\-pagers, readouts, and decks) that clearly communicate: problem statement, approach, progress, risks, decisions needed, and business impact.
- Present technical strategies and delivery status to senior leadership and mixed audiences, adjusting depth while preserving accuracy and decision clarity.
- Own technical “storytelling” for major milestones (architecture approvals, governance checkpoints, production readiness) with crisp visuals and artifacts.
- Define and promote reusable patterns, components, and best practices for AI engineering to accelerate delivery across teams.
- Develop and publish AI standards/best practices and participate in model lifecycle governance (e.g., model registration/curation processes, evaluation and transparency measures).
- Contribute to technology roadmaps and guidance that standardizes delivery and improves operational resilience.
- Mentor engineers, lead design and code reviews, develop communities of practice, and systematically raise AI engineering capability across delivery teams.
- Demonstrates end\-to\-end accountability for the strategy, architecture, standards, roadmap, adoption, and operational maturity of an assigned technical domain. Serves as the organization’s recognized subject\-matter authority while ensuring the domain’s capabilities are reusable, measurable, governed, and effectively adopted across delivery teams.
- Complete work with minimal supervision.
Qualifications
- Bachelor's Degree in Computer Science, Statistics, Engineering or related field, or the equivalent combination of education, training and experience.
- 7\-10 years of experience in AI or similar.
- Experience with modern Generative AI and agentic patterns (e.g., LLM workflows, orchestration frameworks, RAG, grounding, evaluation, safety controls).
- Experience establishing AI standards, best practices, and scalable enablement mechanisms (reference architectures, reusable components).
- Proven track record of driving and coordinating use of GenAI Code Assistants (GitHub Copilot, etc.) to drive Developer Productivity initiatives across the organization with clear value metrics.
- Hands\-on experience building production grade AI agents using industry leading platforms (Azure AI Foundry, etc.) and related technologies such as MCP or A2A.
- Experience with data platforms (Databricks, etc.) and organizing, cataloging and chunking of unstructured data for scalable Generative\-AI solutions and robust knowledge management.
- Experience with vector stores and graph databases for managing complex relationships to use in AI applications such as recommendation systems.
- Robust experience in Azure AI/Data solutions and a deep understanding of the evolving AI landscape, with proven track record in API integrations for accessing LLMs.
- Demonstrated experience conducting threat modeling and implementing security controls for AI systems, including prompt\-injection defenses, data\-loss prevention, agent authorization, secure tool execution, secrets management, and adversarial testing.
- Experience operationalizing Responsible AI and model\-risk requirements through technical controls, evaluation criteria, documentation, human\-in\-the\-loop patterns, traceability, and auditable evidence.
- Advanced software engineering skills in Python and API/service development, with experience designing distributed, resilient, containerized systems and implementing automated testing
- Experience with Agile/SAFe delivery and CI/CD practices for AI solutions.
- Significant experience working with structured and unstructured data.
- Significant experience in developing sophisticated algorithms to automate processes and tasks.
- Advanced knowledge of current AI technologies and concepts.
Additional Information
Hours:
- Monday \- Friday, 8:00AM \- 4:30PM
Location:
- 820 Follin Lane, Vienna, VA 22180
About Us
Navy Federal provides much more than a job. We provide a meaningful career experience, including a culture that is energized, engaged and committed; and fierce appreciation for our teams, who are rewarded with highly competitive pay and generous benefits and perks.
Our approach to careers is simple yet powerful: Make our mission your passion.
- FORTUNE100 Best Companies to Work For® 2025
- Yello and WayUp Top 100 Internship Programs
- Computerworld® Best Places to Work in IT
- Newsweek Most Loved Workplaces
- 2025 PEOPLE® Companies That Care
- Newsweek Most Trustworthy Companies in America
- Military Times 2025 Best for Vets Employers
- Best Companies for Latinos to Work for 2025
- Forbes® 2025 America’s Best Large Employers
- Forbes® 2025 America's Best Employers for New Grads
- Forbes® 2025 America's Best Employers for Tech Workers
- 2025 RippleMatch Campus Forward Award Winner for Overall Excellence
- Military.com Top Military Spouse Employers 2025
- 2025 Handshake Early Talent Award
From *Fortune* . ©2025 *Fortune* Media IP Limited. All rights reserved. Used under license. *Fortune* and *Fortune* Media IP Limited are not affiliated with, and do not endorse products or services of, Navy Federal Credit Union.
Equal Employment Opportunity: All qualified applicants will receive consideration for employment without regard to age, race, sex, color, religion, national origin, disability, veteran status, pregnancy, sexual orientation, genetic information, gender identity or any other basis protected by applicable law.
Accommodations: If you need accommodation or assistance for a qualifying condition to complete the online application (or during any stage of the hiring process), you can contact Navy Federal's Medical Accommodations team at medicalaccommodations@navyfederal.org or by calling 1\-888\-503\-6013\. This team cannot provide any information on job postings or application status.
Disclaimers: Navy Federal reserves the right to fill this role at a higher/lower grade level based on business need. An assessment may be required to compete for this position. Job postings are subject to close early or extend out longer than the anticipated closing date at the hiring team’s discretion based on qualified applicant volume. Navy Federal Credit Union assesses market data to establish salary ranges that enable us to remain competitive. You are paid within the salary range, based on your experience, location and market position. For additional details regarding compensation and benefits, review the Benefits page of the Navy Federal Career Site.
Protect Yourself from Job Scams: Navy Federal Credit Union jobs are posted on our career site, jobs.navyfederal.org and reputable job boards (e.g., LinkedIn, Indeed). We do not post jobs on social media marketplaces, messaging apps or unverified websites. We will never ask candidates for payment, bank details or personal financial information during the hiring process.
Bank Secrecy Act: Remains cognizant of and adheres to Navy Federal policies and procedures, and regulations pertaining to the Bank Secrecy Act.
Role Details
About This Role
AI/ML Engineers build and deploy machine learning models in production. They work across the full ML lifecycle: data pipelines, model training, evaluation, and serving infrastructure. The role has evolved significantly over the past two years. Where ML Engineers once spent most of their time on model architecture, the job now tilts heavily toward inference optimization, cost management, and integrating LLM capabilities into existing systems. Companies want engineers who can ship production systems, and the experimenter-only role is fading fast.
Day-to-day, you're writing training pipelines, debugging data quality issues, setting up evaluation frameworks, and figuring out why your model performs differently in staging than it did on your dev set. The best ML engineers are obsessive about reproducibility and measurement. They instrument everything. They know that a model is only as good as the data feeding it and the infrastructure serving it.
Across the 3,708 AI roles we're tracking, AI/ML Engineer positions make up 70% of the market. At Navy Federal Credit Union, this role fits into their broader AI and engineering organization.
Demand for AI/ML Engineers has been strong and consistent. Unlike some AI roles that spike with hype cycles, ML engineering is a foundational need. Every company deploying AI models needs people who can keep them running, and the gap between research prototypes and production systems keeps growing.
What the Work Looks Like
A typical week might include: debugging a data pipeline that's silently dropping 3% of training examples, running A/B tests on a new model version, writing documentation for a feature flag system that lets you roll back model deployments, and reviewing a junior engineer's PR for a new evaluation metric. Meetings tend to be cross-functional since ML touches product, engineering, and data teams.
Demand for AI/ML Engineers has been strong and consistent. Unlike some AI roles that spike with hype cycles, ML engineering is a foundational need. Every company deploying AI models needs people who can keep them running, and the gap between research prototypes and production systems keeps growing.
Skills Required
Python and PyTorch dominate the requirements. Most roles expect experience with cloud platforms (AWS, GCP, or Azure) and familiarity with ML frameworks like TensorFlow or JAX. RAG (Retrieval-Augmented Generation) has become a top-3 skill requirement as companies integrate LLMs into their products. Docker and Kubernetes show up in about a third of postings, reflecting the production focus of the role.
Beyond the core stack, employers increasingly want experience with experiment tracking tools (MLflow, Weights & Biases), feature stores, and vector databases. Fine-tuning experience is valuable but less common than you'd think from reading Twitter. Most production LLM work is RAG and prompt engineering, not fine-tuning. If you have both, you're in a strong position.
Companies that are serious about AI/ML hiring tend to post specific infrastructure details in the job description: the frameworks they use, their model serving stack, their data pipeline tools. Vague postings that just say 'ML experience required' without specifics are often companies that haven't figured out what they need yet.
Compensation Benchmarks
AI/ML Engineer roles pay a median of $218,750 based on 3,817 positions with disclosed compensation. Senior-level AI roles across all categories have a median of $230,000.
Across all AI roles, the market median is $217,500. Top-quartile compensation starts at $272,100. The 90th percentile reaches $325,000. For comparison, the highest-paying categories include AI Safety ($300,000) and Research Engineer ($280,000). By seniority level: Entry: $120,000; Mid: $200,000; Senior: $230,000; Director: $272,150; VP: $250,000.
Navy Federal Credit Union AI Hiring
Navy Federal Credit Union has 1 open AI role right now. They're hiring across AI/ML Engineer. Based in Vienna, VA, US.
Location Context
Across all AI roles, 14% (508 positions) offer remote work, while 3,180 require on-site attendance. Top AI hiring metros: New York (1,045 roles, $220,000 median); San Francisco (810 roles, $277,088 median); Los Angeles (397 roles, $215,000 median).
Career Path
Common paths into AI/ML Engineer roles include Data Scientist, Software Engineer, Research Engineer.
From here, career progression typically leads toward ML Architect, AI Engineering Manager, Principal ML Engineer.
The fastest path into ML engineering is through software engineering with a self-directed ML education. A CS degree helps, but production engineering skills matter more than academic credentials. Build something that works, deploy it, and measure it. That portfolio project is worth more than a Coursera certificate. For career growth, the fork comes around the senior level: go deep on technical complexity (staff/principal track) or move into managing ML teams.
What to Expect in Interviews
Expect system design questions around ML pipelines: how you'd build a training pipeline for a specific use case, handle data drift, or design A/B testing infrastructure for model deployments. Coding rounds typically involve Python, with emphasis on data manipulation (pandas, numpy) and algorithm implementation. Take-home assignments often ask you to build an end-to-end ML pipeline from raw data to deployed model.
When evaluating opportunities: Companies that are serious about AI/ML hiring tend to post specific infrastructure details in the job description: the frameworks they use, their model serving stack, their data pipeline tools. Vague postings that just say 'ML experience required' without specifics are often companies that haven't figured out what they need yet.
AI Hiring Overview
The AI job market has 3,708 open positions tracked in our dataset. By seniority: 102 entry-level, 1,705 mid-level, 1,469 senior, and 432 leadership roles (Director, VP, C-Level). Remote roles make up 14% of the market (508 positions). The remaining 3,180 roles require on-site or hybrid attendance.
The market median for AI roles is $217,500. Top-quartile compensation starts at $272,100. The 90th percentile reaches $325,000. Highest-paying categories: AI Safety ($300,000 median, 21 roles); Research Engineer ($280,000 median, 147 roles); AI Architect ($254,798 median, 67 roles).
Demand for AI/ML Engineers has been strong and consistent. Unlike some AI roles that spike with hype cycles, ML engineering is a foundational need. Every company deploying AI models needs people who can keep them running, and the gap between research prototypes and production systems keeps growing.
The AI Job Market Today
The AI job market spans 3,708 open positions across 16 role categories. The largest categories by volume: AI/ML Engineer (2,605), Data Scientist (310), AI Software Engineer (259). These three account for the majority of open positions, though smaller categories often have higher per-role compensation because of specialized skill requirements.
The seniority mix tells a story about where AI teams are in their maturity. Entry-level roles (102) are outnumbered by mid-level (1,705) and senior (1,469) positions, reflecting that most companies are past the 'build a team from scratch' phase and need experienced engineers who can ship production systems. Leadership roles (Director, VP, C-Level) total 432 positions, representing the bottleneck between technical execution and organizational strategy.
Remote work availability sits at 14% of all AI roles (508 positions), with 3,180 requiring on-site or hybrid attendance. The remote share has stabilized after the post-pandemic correction. Senior and specialized roles (Research Scientist, ML Architect) are more likely to be remote-eligible than entry-level positions, partly because experienced hires have more negotiating power and partly because these roles require less hands-on mentorship.
AI compensation is structured in clear tiers. The market median sits at $217,500. Top-quartile roles start at $272,100, and the 90th percentile reaches $325,000. These figures include base salary with disclosed compensation. Total compensation (including equity, bonuses, and sign-on) runs 20-40% higher at companies that offer those components.
Category matters for compensation. AI Safety roles lead at $300,000 median, while Prompt Engineer roles sit at $140,000. The spread between highest and lowest-paying categories reflects the premium on specialized technical skills versus broader analytical roles.
The most in-demand skills across all AI postings: Python (1,890 postings), Aws (1,103 postings), Azure (877 postings), Rag (855 postings), Gcp (631 postings), Prompt Engineering (560 postings), Pytorch (545 postings), Claude (498 postings). Python dominates, appearing in the vast majority of role descriptions regardless of category. Cloud platform experience (AWS, GCP, Azure) is the second most common requirement. The newer entrants to the top skills list (RAG, vector databases, LLM APIs) reflect the shift from traditional ML toward generative AI applications.
Frequently Asked Questions
Get Weekly AI Career Intelligence
Salary data, skills demand, and market signals from 16,000+ AI job postings. Every Monday.