Finance Model Prompt Evaluator
Role Overview
We are seeking expert finance and economics professionals to author and verify high-quality open-ended prompts for AI model evaluation. You will craft and review challenging, unambiguous financial analysis problems across core subdomains, assessing AI reasoning quality and helping establish rigorous evaluation standards for frontier language models. You will be assigned one of two task types:
- *Authoring Task**
Create 5 original, open-ended prompts from your assigned subdomain at varying difficulty levels (undergraduate, advanced undergraduate, or graduate/professional). Prompts should require human judgment to evaluate the quality of the AI's response, such as quantitative analysis, risk modeling, or regulatory reasoning.
- *Verification Task**
Review 5 authored prompts for clarity, scope alignment, difficulty accuracy, and uniqueness. Edit prompts and difficulty ratings where needed.
- *Financial Analysis Subdomains Covered**
Quantitative Finance, Derivatives & Trading; Macroeconomics, Rates, FX & Sovereign Finance; Banking, Lending & Financial Institutions; Risk Management, ALM & Insurance; Wealth Management, Personal Finance, Digital & Alternative Assets; Real Estate, Infrastructure, Commodities & Tangible Assets; Regulation, Compliance, Tax & Cross-Border Structuring.
- *Key Responsibilities**
- Author clear, unambiguous, open-ended financial prompts that elicit evaluable AI responses - Verify prompts are within the scope of the assigned subdomain and correctly rated for difficulty - Ensure all 5 prompts in a task are sufficiently distinct from one another with varying difficulty levels - Apply expert judgment to assess the depth and quality of financial reasoning required - Edit prompts and difficulty assignments where standards are not met
- *Ideal Qualifications**
- Master's degree or higher in Finance, Economics, Financial Engineering, or a closely related field - 2–6 years of professional experience in financial services, investment banking, asset management, or a related field - Strong command of financial modeling, quantitative methods, and domain-specific regulatory frameworks - CFA, FRM, CPA, or equivalent professional certification is a strong plus - Excellent written English and ability to craft precise, well-scoped technical questions
- *More About the Opportunity**
- Expected commitment: 10+ hours/week - Asynchronous, fully remote work Apply tot his job Apply To this Job