Senior Software Engineer – Python ( LLM Evaluation & Repository Validation)

Analyze and triage GitHub issues across trending open-source libraries.

Set up and configure code repositories, including Dockerization and environment setup.

Evaluating unit test coverage and quality.

Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.

Collaborate with researchers to design and identify repositories and issues that are challenging for LLMs.

Opportunities to lead a team of junior engineers to collaborate on projects.

Required Skills:

Strong experience with Java,

Proficiency with Git, Docker, and basic software pipeline setup.

Ability to understand and navigate complex codebases.

Comfortable running, modifying, and testing real-world projects locally.

Experience contributing to or evaluating open-source projects is a plus.

Nice to Have:

Previous participation in LLM research or evaluation projects.

Experience building or testing developer tools or automation agents.