Join to apply for the Data Analyst (Remote) role at AI Jobs
We are looking for a data-driven professional to join our analytics and quality team. In this role, you will analyze the performance of AI agents across finance‑sector tasks to uncover root causes of failure and drive improvements in design, evaluation, and benchmarking frameworks.
Key Responsibilities
- Conduct statistical failure analysis to identify recurring patterns in AI agent performance across prompts, rubrics, templates, and file types
- Investigate root causes behind performance issues in task execution, framework design, or data complexity
- Perform multi-dimensional analysis across finance sub‑domains, task categories, and input types
- Develop dashboards and reports highlighting performance trends, failure clusters, and improvement opportunities
- Recommend enhancements to task design, quality rubrics, and evaluation metrics based on data findings
Required Qualifications
- Strong understanding of statistical analysis, hypothesis testing, and pattern recognition
- Proficiency in Python (pandas, scipy, matplotlib/seaborn) or R for data analytics
- Hands‑on experience in exploratory data analysis (EDA) and turning data into actionable insights
- Familiarity with AI/ML evaluation techniques and quality assessment metrics
- Proficiency in Excel, SQL, and visualization tools such as Tableau or Looker
Preferred Qualifications
- Prior experience in AI/ML model evaluation, data labeling quality, or automation analysis
- Knowledge of finance‑sector concepts or willingness to learn finance domain structures
- Experience with multi‑dimensional failure analysis or benchmarking studies
- Understanding of large‑scale evaluation frameworks or test dataset design
We welcome applicants from all backgrounds. Reasonable accommodations are available for qualified individuals with disabilities.
Job Details
- Seniority level: Not Applicable
- Employment type: Part‑time
- Job function: Information Technology
- Industries: IT System Data Services and Technology, Information and Media
Referrals increase your chances of interviewing at AI Jobs by 2x
#J-18808-Ljbffr