Interested in this AI/ML Engineer role at Commonwealth of Massachusetts?
Apply Now →Skills & Technologies
About This Role
SALARY RANGE (Grade 15) CSA216: $89,993.11/yr. - $134,991.01/yr.(Commensurate with experience)
GENERAL STATEMENT OF DUTIES:
The Senior AI Engineer designs the architecture for, builds, and deploys AI-enabled applications in OSA's Microsoft GCC environment. Working with the Innovation Assistant Director and Cloud Engineering, the role shapes OSA's AI reference architecture and turns it into production Retrieval-Augmented Generation (RAG) applications, agent workflows, the data pipelines that feed them, and cost- and performance-aware model deployments. The role builds applications on existing foundation models accessed through approved platforms such as Azure AI Foundry; it does not train or develop LLMs. The primary focus is the Audit division, delivering AI services that measurably improve audit planning, research, and analysis across OSA's statewide audit portfolio. The role also builds AI-enabled programs for OSA's administrative and operational units, including Human Resources, Administration & Finance (A&F), Occupational Schools, the Division of Local Mandates (DLM), and the Bureau of Special Investigations (BSI).
OSA does not expect candidates to know every tool named in this description. Solid architecture and engineering fundamentals and hands-on experience delivering LLM applications matter most.
SUPERVISION RECEIVED:
Reports to the Innovation Assistant Director and works within OSA's IT Innovation group. Collaborates closely with the Manager, System and Process Improvements, Cloud Engineering (e.g., platform security, IaC, APIM/Entra), Enterprise IT, the Data Analytics Unit, data engineering staff, contractors, and business stakeholders across Audit and OSA's operational units. The Manager, System and Process Improvements maintains the backlog of Audit process-improvement opportunities, designs and measures pilots of AI-assisted audit process steps and recommends adoption; Audit and agency leadership decide whether to adopt changes to audit processes. The Innovation Assistant Director runs intake for OSA's other units, sets IT Innovation work priorities in line with the IT Governance Council, manages staff and contractors, and decides technology and build-versus-buy choices. This role owns the technical design and engineering of the solutions and does not supervise staff.
DUTIES AND RESPONSIBILITIES:
AI Solution Architecture
- Design solution architectures for AI applications — components, integrations, data flows, security boundaries, and deployment topology — and document them in diagrams and architecture decision records.
- Lead the definition and maintenance of OSA's AI reference architecture, including standard patterns for RAG, agents, API exposure, logging, and evaluation, with review and approval by the Innovation Assistant Director.
- Work with Cloud Engineering to extend Azure Landing Zone patterns (CAF/WAF) for AI workloads: subscriptions, Azure Policy, VNets/NSGs, private endpoints, Key Vault, and managed identities.
- Define microservice and API boundaries (REST, APIM) so AI capabilities are reused across audit and operational applications rather than rebuilt for each one.
- Lead technical design reviews, and provide technical assessments of new Azure AI services, vendor tools, and build-versus-buy options to inform technology decisions by the Innovation Assistant Director.
- Design for cost from the start: model and hosting choices, budgets, and tagging so AI consumption is visible and attributable by project.
Software Development
- Write, test, and maintain production Python and TypeScript/JavaScript code for AI-enabled applications, services, and user interfaces.
- Build REST APIs and backend services that connect LLMs to OSA documents, data sources, and business systems.
- Write automated unit, integration, and evaluation tests; set code review and Git/GitHub branching standards for AI projects.
- Build the automated tests, evaluation harnesses, and quality metrics used in OSA's software quality assurance process for AI applications (distinct from the audit Quality Assurance function), which data engineering staff and testers use to carry out testing.
- Use AI-assisted development tools such as GitHub Copilot responsibly, reviewing generated code for correctness and security.
- Troubleshoot and improve the AI components of applications running in production.
AI Application Engineering
- Own retrieval design for RAG applications — hierarchical/semantic chunking, embedding choices, hybrid (keyword + vector) search, cross-encoder reranking, and citation fidelity checks — working with data engineering staff who build ingestion, schema, and embedding pipelines.
- Engineer agent and tool-calling workflows with Microsoft Agent Framework; lead migration of existing Prompt Flow orchestration and evaluation into supported pathways.
- Define and instrument quality and performance measures — groundedness, relevance, citation accuracy (e.g., RAGAS), latency, TTFT, tokens/sec, concurrency, and cost-per-query — with dashboards in Azure Monitor, Log Analytics, and Application Insights, and supply usage and adoption data for the audit workflow KPIs and dashboard maintained by the Manager, System and Process Improvements.
- Build the technical components of bounded pilots designed by the Manager, System and Process Improvements, and provide the data needed to measure results against established baselines.
- Translate business requirements — developed with the Manager, System and Process Improvements for Audit or with the Innovation Assistant Director for other units — into technical requirements, working prototypes, and production applications.
Platform, Deployment & Operations
- Containerize services with Docker and deploy them on Kubernetes (AKS), including health checks, autoscaling, secrets integration, and rolling updates.
- Provide ongoing support for the AI layer of deployed applications — models, prompts, retrieval, agents, and evaluations — and transition hosting, infrastructure, and general application support to Enterprise IT with runbooks and knowledge transfer.
- Build and maintain infrastructure as code (Terraform or Bicep) and GitHub Actions CI/CD pipelines, and design Dev/Test/Prod and sandbox environment patterns, including promotion standards for AI services, prompts, model configurations, and evaluation assets.
- Configure AI Foundry, APIM, Entra ID app registrations, and related Azure services for secure-by-default deployments.
- Design backup, recovery, and business continuity approaches for AI services and document them in runbooks for Enterprise IT.
Data for AI
- Design ingestion and transformation pipelines (e.g., Azure Data Factory, Python, SQL, dbt), and set the patterns data engineering staff use to build and maintain them, to prepare audit, agency, and internal operational data for retrieval indexes, knowledge graphs, and analytics.
- Define data interface contracts — schemas, validation rules, and reconciliation checks — and build pipelines with retries, backfills, and idempotency.
Model Selection & Performance
- Select and configure model deployments (e.g., standard vs provisioned throughput, regional availability in GCC) based on capability, latency, capacity, and cost for each use case.
- Evaluate provider and hosting options, including the underlying accelerators (NVIDIA H100/H200/B200; AMD MI300; LPU-based services), when comparing throughput, latency, and cost.
- Apply application-level optimizations: prompt/prefix caching, batch and streaming APIs, context and token budgeting, and routing between larger and smaller models.
Knowledge Graphs & Taxonomy
- Integrate OSA's taxonomy (categories defined with the Data Analytics Unit) and entity relationships, managed in Synaptica Graphite, into retrieval and agent workflows, and expose graph-powered APIs for Audit Research Tool, Audit Search, and TeamMate+.
- Model domain ontologies (entities, events, regulations) to support retrieval, entity linking, and policy-aware reasoning within agents.
Security, Privacy & Governance
- Design least-privilege access using Entra ID security group patterns (team vs access), RBAC, and PIM, and implement OSA AI policies in code (policy-as-code).
- Implement audit-ready logging (prompts, tool calls, citations), FOIA-aware retention, and privacy controls such as DLP labels and redaction filters.
- Build application features that support GAGAS-compatible protocols for documenting AI assistance, human verification, and supervisory review, as defined by the Manager, System and Process Improvements with Quality Assurance.
- Protect sensitive data (PII, PHI, HIPAA-regulated healthcare data) with encryption at rest and in transit, key management, and classification-driven access.
- Partner with cybersecurity on threat modeling and security reviews, and maintain audit-ready evidence of controls.
Delivery & Collaboration
- Share technical knowledge with developers and analysts through code reviews, documentation, and pairing on AI agent patterns, retrieval engineering, and reproducible deployments.
- Build AI-enabled applications, as prioritized by the Innovation Assistant Director, for Human Resources, Administration & Finance (A&F), Occupational Schools, the Division of Local Mandates (DLM), and the Bureau of Special Investigations (BSI), reusing shared platform components.
- Provide day-to-day technical direction to the AI Application Developer and data engineering staff on application, pipeline, schema, and retrieval work.
- Produce architecture documentation and operational runbooks that let Enterprise IT and OSA staff operate and extend systems without vendor dependency.
- Explain architecture choices and trade-offs to the IT Innovation team and project stakeholders.
The Office of the State Auditor is committed to providing equal employment opportunities. Employment actions such as recruiting, hiring, training, and promoting individuals are based upon a policy of non-discrimination. Employment decisions and actions are made without regard to race, color, gender, religion, age, national origin, ancestry, sexual orientation, gender identity and expression, disability, military status, genetic information, political affiliation, or veteran’s status.
Qualifications
MINIMUM QUALIFICATIONS:
The successful candidate will possess and/or demonstrate:
- Bachelor’s degree in computer science, software engineering, or a related field, or equivalent experience.
- 5–7 years of professional software engineering experience, including at least 2 years building production applications that use LLMs (e.g., RAG, agents, or tool-calling features).
- Demonstrated design ownership of the architecture for at least one production system, including documentation of design decisions.
- Strong Python; working proficiency in TypeScript/JavaScript and SQL.
- Hands-on building RAG systems with hybrid search and reranking; experience with embeddings and vector stores (e.g., Azure AI Search, pgvector, Qdrant, Weaviate).
- Experience designing and deploying applications on Azure (or AWS/GCP with a willingness to work in Azure), including identity, private networking, and secrets management.
- Experience with Docker and Kubernetes, infrastructure as code (Terraform or Bicep), and CI/CD pipelines.
- Experience designing REST APIs and service boundaries, and writing automated tests.
- Experience handling sensitive data (PII/PHI) under least-privilege access and encryption controls.
- Clear written communication, including architecture documentation, and the ability to explain technical trade-offs.
PREFERRED QUALIFICATIONS:
- Azure Government/Microsoft GCC experience, including Azure Landing Zones and Cloud Adoption Framework implementation.
- Microsoft Azure certifications at the associate or expert level (e.g., AI engineering or solutions architecture).
- Microsoft Agent Framework and Prompt Flow migration; LangChain/LangGraph; structured outputs and function/tool calling across providers.
- Deeper Kubernetes experience on AKS: Helm charts, ingress, autoscaling, and workload identity.
- Knowledge graphs, taxonomies, or ontologies (e.g., RDF, SPARQL); familiarity with Synaptica Graphite.
- Data engineering with Databricks (Unity Catalog, Delta Lake) or Microsoft Fabric; data governance frameworks (e.g., DMBOK), metadata and lineage management, and MDM.
- RBAC/ABAC design, Purview classification, DLP labels, Defender for Cloud, and policy-as-code.
- Understanding of LLM serving trade-offs: quantization (FP8/INT4), KV-cache strategies, MoE models and expert parallelism, and open-weight serving stacks (vLLM, TensorRT-LLM, SGLang).
- Experience in government, healthcare, or financial services, including shared platform or data standards across multiple agencies or business units.
- Experience integrating TeamMate+ and external sources into AI-driven research workflows.
- Experience mentoring developers or serving as technical lead on a project.
- Exposure to AWS or GCP data and AI services (e.g., Vertex AI).
Salary is commensurate with experience.
The Office of the State Auditor is committed to providing equal employment opportunities. Employment actions such as recruiting, hiring, training, and promoting individuals are based upon a policy of non-discrimination. Employment decisions and actions are made without regard to race, color, gender, religion, age, national origin, ancestry, sexual orientation, gender identity and expression, disability, military status, genetic information, political affiliation, or veteran’s status.
Official Title: Applications Manager Primary Location
: United States-Massachusetts-Boston-1 Ashburton Place Job
: Information Systems and Technology Agency
: Office of the State Auditor Schedule
: Full-time Shift
: Day Job Posting
: Oct 29, 2026, 1:37:04 PM Number of Openings
: 1 Salary
: 89,993.11 - 134,991.01 Yearly If you have Diversity, Affirmative Action or Equal Employment Opportunity questions or need a Reasonable Accommodation, please contact Diversity Officer / ADA Coordinator: Marie Tedesco - 6177277200 Potentially Eligible for a Hybrid Work Schedule: Yes
Salary Context
This $89K-$134K range is in the lower quartile for AI/ML Engineer roles in our dataset (median: $180K across 1953 roles with salary data).
View full AI/ML Engineer salary data →Role Details
About This Role
AI/ML Engineers build and deploy machine learning models in production. They work across the full ML lifecycle: data pipelines, model training, evaluation, and serving infrastructure. The role has evolved significantly over the past two years. Where ML Engineers once spent most of their time on model architecture, the job now tilts heavily toward inference optimization, cost management, and integrating LLM capabilities into existing systems. Companies want engineers who can ship production systems, and the experimenter-only role is fading fast.
Day-to-day, you're writing training pipelines, debugging data quality issues, setting up evaluation frameworks, and figuring out why your model performs differently in staging than it did on your dev set. The best ML engineers are obsessive about reproducibility and measurement. They instrument everything. They know that a model is only as good as the data feeding it and the infrastructure serving it.
Across the 10,419 AI roles we're tracking, AI/ML Engineer positions make up 75% of the market. At Commonwealth of Massachusetts, this role fits into their broader AI and engineering organization.
Demand for AI/ML Engineers has been strong and consistent. Unlike some AI roles that spike with hype cycles, ML engineering is a foundational need. Every company deploying AI models needs people who can keep them running, and the gap between research prototypes and production systems keeps growing.
What the Work Looks Like
A typical week might include: debugging a data pipeline that's silently dropping 3% of training examples, running A/B tests on a new model version, writing documentation for a feature flag system that lets you roll back model deployments, and reviewing a junior engineer's PR for a new evaluation metric. Meetings tend to be cross-functional since ML touches product, engineering, and data teams.
Demand for AI/ML Engineers has been strong and consistent. Unlike some AI roles that spike with hype cycles, ML engineering is a foundational need. Every company deploying AI models needs people who can keep them running, and the gap between research prototypes and production systems keeps growing.
Skills Required
Python and PyTorch dominate the requirements. Most roles expect experience with cloud platforms (AWS, GCP, or Azure) and familiarity with ML frameworks like TensorFlow or JAX. RAG (Retrieval-Augmented Generation) has become a top-3 skill requirement as companies integrate LLMs into their products. Docker and Kubernetes show up in about a third of postings, reflecting the production focus of the role.
Beyond the core stack, employers increasingly want experience with experiment tracking tools (MLflow, Weights & Biases), feature stores, and vector databases. Fine-tuning experience is valuable but less common than you'd think from reading Twitter. Most production LLM work is RAG and prompt engineering, not fine-tuning. If you have both, you're in a strong position.
Companies that are serious about AI/ML hiring tend to post specific infrastructure details in the job description: the frameworks they use, their model serving stack, their data pipeline tools. Vague postings that just say 'ML experience required' without specifics are often companies that haven't figured out what they need yet.
Compensation Benchmarks
AI/ML Engineer roles pay a median of $213,800 based on 16,346 positions with disclosed compensation. Senior-level AI roles across all categories have a median of $230,000. This role's midpoint ($112K) sits 47% below the category median. Disclosed range: $89K to $134K.
Across all AI roles, the market median is $214,834. Top-quartile compensation starts at $266,000. The 90th percentile reaches $321,730. For comparison, the highest-paying categories include AI Safety ($274,500) and Research Engineer ($252,000). By seniority level: Entry: $114,400; Mid: $193,200; Senior: $230,000; Director: $272,100; VP: $235,000.
Commonwealth of Massachusetts AI Hiring
Commonwealth of Massachusetts has 1 open AI role right now. They're hiring across AI/ML Engineer. Based in Boston, MA, US. Compensation range: $134K - $134K.
Location Context
AI roles in Boston pay a median of $215,000 across 484 tracked positions.
Career Path
Common paths into AI/ML Engineer roles include Data Scientist, Software Engineer, Research Engineer.
From here, career progression typically leads toward ML Architect, AI Engineering Manager, Principal ML Engineer.
The fastest path into ML engineering is through software engineering with a self-directed ML education. A CS degree helps, but production engineering skills matter more than academic credentials. Build something that works, deploy it, and measure it. That portfolio project is worth more than a Coursera certificate. For career growth, the fork comes around the senior level: go deep on technical complexity (staff/principal track) or move into managing ML teams.
What to Expect in Interviews
Expect system design questions around ML pipelines: how you'd build a training pipeline for a specific use case, handle data drift, or design A/B testing infrastructure for model deployments. Coding rounds typically involve Python, with emphasis on data manipulation (pandas, numpy) and algorithm implementation. Take-home assignments often ask you to build an end-to-end ML pipeline from raw data to deployed model.
When evaluating opportunities: Companies that are serious about AI/ML hiring tend to post specific infrastructure details in the job description: the frameworks they use, their model serving stack, their data pipeline tools. Vague postings that just say 'ML experience required' without specifics are often companies that haven't figured out what they need yet.
AI Hiring Overview
The AI job market has 10,419 open positions tracked in our dataset. By seniority: 117 entry-level, 4,975 mid-level, 3,840 senior, and 1,487 leadership roles (Director, VP, C-Level). Remote roles make up 6% of the market (580 positions). The remaining 9,776 roles require on-site or hybrid attendance.
The market median for AI roles is $214,834. Top-quartile compensation starts at $266,000. The 90th percentile reaches $321,730. Highest-paying categories: AI Safety ($274,500 median, 90 roles); Research Engineer ($252,000 median, 580 roles); AI Engineering Manager ($250,811 median, 72 roles).
Demand for AI/ML Engineers has been strong and consistent. Unlike some AI roles that spike with hype cycles, ML engineering is a foundational need. Every company deploying AI models needs people who can keep them running, and the gap between research prototypes and production systems keeps growing.
The AI Job Market Today
The AI job market spans 10,419 open positions across 16 role categories. The largest categories by volume: AI/ML Engineer (7,849), AI Product Manager (516), AI Software Engineer (513). These three account for the majority of open positions, though smaller categories often have higher per-role compensation because of specialized skill requirements.
The seniority mix tells a story about where AI teams are in their maturity. Entry-level roles (117) are outnumbered by mid-level (4,975) and senior (3,840) positions, reflecting that most companies are past the 'build a team from scratch' phase and need experienced engineers who can ship production systems. Leadership roles (Director, VP, C-Level) total 1,487 positions, representing the bottleneck between technical execution and organizational strategy.
Remote work availability sits at 6% of all AI roles (580 positions), with 9,776 requiring on-site or hybrid attendance. The remote share has stabilized after the post-pandemic correction. Senior and specialized roles (Research Scientist, ML Architect) are more likely to be remote-eligible than entry-level positions, partly because experienced hires have more negotiating power and partly because these roles require less hands-on mentorship.
AI compensation is structured in clear tiers. The market median sits at $214,834. Top-quartile roles start at $266,000, and the 90th percentile reaches $321,730. These figures include base salary with disclosed compensation. Total compensation (including equity, bonuses, and sign-on) runs 20-40% higher at companies that offer those components.
Category matters for compensation. AI Safety roles lead at $274,500 median, while Prompt Engineer roles sit at $140,000. The spread between highest and lowest-paying categories reflects the premium on specialized technical skills versus broader analytical roles.
The most in-demand skills across all AI postings: Python (2,350 postings), Aws (1,173 postings), Azure (1,030 postings), Rag (986 postings), Gcp (714 postings), Pytorch (649 postings), Kubernetes (583 postings), Claude (581 postings). Python dominates, appearing in the vast majority of role descriptions regardless of category. Cloud platform experience (AWS, GCP, Azure) is the second most common requirement. The newer entrants to the top skills list (RAG, vector databases, LLM APIs) reflect the shift from traditional ML toward generative AI applications.
Frequently Asked Questions
Get Weekly AI Career Intelligence
Salary data, skills demand, and market signals from 16,000+ AI job postings. Every Monday.