Note: This position is open exclusively to candidates currently enrolled in the Hiring Our Heroes Fellows Program. Applications from individuals not participating in the program will not be considered.
Skills & Qualifications
Required Skills & Qualifications
- Understanding of the business function and M&Ps; align AI behavior with policy and process; plus knowledge of supported workflows and tools with experience operating the systems involved (e.g., ticketing/workflow platforms).
- Strong analytical and problem-solving skills with meticulous attention to detail; proven root cause analysis (RCA) capability.
- General AI literacy and understanding of agentic systems; basic prompt engineering (iteration, testing, versioning).
- Ability to manage fallout within SLAs, triage tickets, and drive rapid resolution; strong prioritization in fast-paced environments.
- Observability: proficiency with logs, metrics, dashboards, and alerts; define and track quality KPIs (accuracy, fallout rate, MTTR).
- Basic scripting understanding to automate corrections, content re-ingestion, and validation workflows.
- Knowledge base authoring and maintenance; clear documentation of training methods, resolutions, and changes for auditability.
- Compliance/data privacy/ethical guidelines awareness; maintain auditable processes and change logs.
- Effective communication: synthesize findings, report metrics, and present recommendations to stakeholders.
Preferred Skills
- Advanced observability (distributed tracing, SLO/SLA design) and incident response practices.
- Experiment tracking and ML operations tooling, feature flags, canary/rollback strategies.
- Familiarity with fine-tuning pipelines, retrieval/RAG, vector databases, and content ingestion pipelines.
- SQL/BI tools for advanced analytics and dashboarding; ability to build executive-ready reports.
- Version control (Git) for prompts, KB content, and evaluation artifacts; change management discipline.
- Workflow orchestration for scheduled re-ingestion, evaluations, and reporting.
- Experience in training, quality assurance, documentation, or knowledge management, including taxonomy/ontology design.
- Advanced scripting/automation and experience writing/maintaining Markdown-based runbooks and KB articles.
- Prior experience with AI in production settings and A/B testing platforms.