Automated evaluation of AI responses on legal documents

This automation template enables objective assessment of AI assistant outputs in the legal domain by comparing them against source documents. The system evaluates factual accuracy, relevance, and completeness—critical for working with regulations, contracts, and case law.

## Who it´s for
- AI quality evaluators in the legal sector
- Teams testing LLMs for factual reliability
- Developers building automated AI response verification systems
- Legal startups using AI for document analysis

## What the automation does
- Receives test data via HTTP webhook (AI query, response, document link)
- Fetches a PDF from Google Drive using a link stored in Google Sheets
- Extracts text from the PDF and sends it with input/output to an LLM judge
- Evaluates the AI response using a LangChain agent and LLM via OpenRouter against strict criteria
- Structures the verdict (Pass/Fail) with reasoning into JSON format
- Saves the result back to Google Sheets for tracking and model comparison

## What´s included
- Ready-to-use n8n workflow
- Trigger and handler logic based on HTTP webhook
- Integrations with Google Sheets, Google Drive, OpenRouter, and HTTP API
- Basic textual guide for launch and adaptation

## Requirements for setup
- n8n instance with external API access
- Google account with Google Sheets and Google Drive APIs enabled
- OpenRouter API key for LLM access
- Access to test data: AI queries, responses, and PDF documents

## Benefits and outcomes
- Objective, repeatable AI quality scoring without human bias
- Detection of hallucinations and factual errors in legal reasoning
- Ability to benchmark multiple LLMs on identical legal tasks
- Automatic accumulation of test results in Google Sheets for performance analysis
- Time savings during regression testing of AI assistants
- Support for AI system certification or audit processes in law

## Important: template only
Important: you are purchasing a ready-made automation workflow template only. Rollout into your infrastructure, connecting specific accounts and services, 1:1 setup help, custom adjustments for non-standard stacks and any consulting support are provided as a separate paid service at an individual rate. To discuss custom work or 1:1 help, contact via Telegram: @gleb923.