About Definitive Healthcare:

At Definitive Healthcare, our mission is simple: to connect with the real people who make healthcare happen. With real data on real people, our customers get the Definitive picture of the Healthcare market to move their businesses and organizations forward.

We’re committed to delivering the industry’s most reliable, actionable, and differentiated insights and empowering the innovators and organizations transforming healthcare. Because when our customers succeed, healthcare improves—for everyone.

Since our founding in 2011, we’ve grown into a global company headquartered just outside Boston, Massachusetts, supporting more than 2,100 customers around the world. Along the way, we’ve built a culture centered on purposeful work, collaboration, curiosity, and meaningful impact.

We’re passionate about the work we do and the people we do it with. Through programs like Definitive Cares and our employee-led affinity groups, we create opportunities for our teams to connect and give back to the communities where we live and work.

About the position

This role independently owns the build and evaluation of AI solutions for an assigned use case, operating within the team's established evaluation framework and practices. The Senior Data Scientist exercises professional judgment on technical approach, issue diagnosis, and implementation within their assigned project, escalating decisions that would affect the team's overall evaluation strategy, span multiple projects, or carry material risk. Its impact is measured by the quality and reliability of the systems and evidence it delivers for its assigned use case.

What you’ll do

Own assigned workflows: Independently build, test, and improve LLM, RAG, and tool-using agent workflows for an assigned use case, using maintainable, tested Python code.
Apply the evaluation framework: Select and apply evaluation criteria, baselines, and test datasets within the team's established evaluation approach; does not set that approach for the team.
Select technical approaches: Compare models, embeddings, and retrieval methods for the assigned use case using evidence on quality, reliability, latency, and cost; document recommendations and tradeoffs for that project.
Diagnose and resolve issues: Investigate retrieval misses, incorrect answers, and workflow failures within the assigned project; independently determine root cause and implement and validate the fix.
Maintain and extend test assets: Build and update test cases, benchmark datasets, and regression checks for the assigned use case.
Support production reliability: Monitor and investigate performance changes for workflows they own, and recommend and implement improvements within their project.
Collaborate on delivery: Communicate findings and technical tradeoffs on the assigned project to immediate engineering, product, and domain partners.

What you'll need

Education and experience

Bachelor's degree required; master's degree in a quantitative or technical field preferred.
5+ years of relevant experience in data science, machine learning, applied AI, software engineering, or a related field.
Hands-on experience building and evaluating LLM applications.

Technical and professional skills

Strong Python programming, software engineering and testing, and analytical problem-solving skills.
Strong knowledge of LLMs, embeddings, semantic search, retrieval systems, and RAG architectures.
Strong hands-on experience with Databricks, including Delta Lake, Unity Catalog, Lakebase, Lakeflow Jobs, and MLflow, for building and deploying scalable, production-grade data and AI solutions.
Ability to design representative evaluations for an assigned use case, analyze errors, and use results to improve a system.
Ability to weigh quality, reliability, latency, and cost when recommending an approach for their own project.
Clear communication and effective collaboration with technical and nontechnical partners.
Experience with agent frameworks and AI evaluation platforms.
Experience with healthcare data is preferred, but not required.