Generative AI Engineer

Prompt Engineer / LLM Engineer – Python & Text-to-Query

Company: Algoworks

Location: Latin America (LATAM) – Fully Remote

Argentina, Uruguay, Chile, Costa Rica, Brazil, Colombia and Ecuador

Employment Type: Contract

Experience: 2+ years of Python experience

Work Arrangement: Remote

About Algoworks

Algoworks is a global technology services company helping businesses achieve their goals through innovative digital solutions, software engineering, and emerging technologies. Our teams work across modern technologies to solve complex business challenges and deliver meaningful outcomes for clients.

Role Overview

Algoworks is seeking a hands-on Prompt Engineer / LLM Engineer to evolve an existing natural-language query prototype into a reliable, production-ready feature.

The solution uses a domain-specific prompt and a user's natural-language question to generate structured intent, which is then converted into a Solr query. The LLM does not access row-level or protected data, and the generated query is displayed to the user for transparency.

You will work closely with the client's technical lead to improve prompt quality, expand query coverage, build evaluation frameworks, and develop the Python components required to deliver a safe, explainable, and measurable solution.

Key Responsibilities

• Design, restructure, maintain, and version prompts, including domain taxonomies, query grammar, examples, and guardrails.
• Build and maintain an evaluation harness using real-world user questions to measure accuracy, latency, and cost.
• Expand support for different question types, ambiguous phrasing, domain-specific synonyms, and newly searchable fields.
• Develop Python components for prompt assembly, structured intent parsing, validation, query generation, and automated evaluation.
• Improve explainability by making generated queries, uncertainty, and refusal scenarios understandable to end users.
• Evaluate LLMs from multiple providers and compare alternative approaches, including decision-style models and smaller fine-tuned models.
• Validate generated queries to ensure they follow defined rules and execute safely.
• Establish regression testing and evidence-based evaluation for prompt, model, and system changes.
• Collaborate with the client's technical lead to deliver improvements aligned with technical and business requirements.

Required Skills and Qualifications

• Hands-on prompt engineering experience in a shipped or production product , beyond experimentation or proof-of-concept development.
• Strong Python development skills, with at least 2 years of experience.
• Experience building natural-language-to-query solutions, such as Text-to-SQL, Solr, Elasticsearch, or similar systems.
• Experience with structured LLM outputs, few-shot prompting, retrieval-augmented prompting, and domain vocabulary handling.
• Experience integrating LLM APIs from at least two providers.
• Understanding of query validation, safe execution, error handling, and output validation.
• Experience designing evaluation frameworks to measure accuracy, latency, cost, and regression performance.
• Understanding of LLM context limits, model drift, and production performance trade-offs.
• Ability to design and explain architectures that prevent LLMs from accessing protected or row-level data.

Preferred Qualifications

• Experience with Apache Solr query construction, search, and faceting.
• Familiarity with healthcare or EMS terminology, such as NEMSIS, ICD, or RxNorm.
• Experience fine-tuning small language models or evaluating alternative model architectures.
• Familiarity with LLM evaluation frameworks.
• Exposure to Azure AI, Microsoft Fabric, or related Microsoft AI technologies.

What We’re Looking For

We are looking for a practical engineer who can take an existing LLM-based prototype and improve it into a measurable, maintainable, and production-ready feature. The ideal candidate combines prompt engineering, Python development, search-query generation, systematic evaluation, and a strong understanding of secure AI architecture.

Interested candidates: Apply through LinkedIn or share your updated resume.

Company: Algoworks

Website: https://www.algoworks.com