Agent skill

langchain4j-testing-strategies

Provides unit test, integration test, and mock AI patterns for LangChain4j applications. Creates mock LLM responses, tests retrieval chains, validates RAG workflows, and implements Testcontainers-based integration tests for Java AI services. Use when unit testing AI services, integration testing LangChain4j components, mocking AI models, or testing LLM-based Java applications.

View SKILL.md on GitHub Repository
Stars 192
Forks 20

Install this agent skill to your Project

npx add-skill https://github.com/giuseppe-trisciuoglio/developer-kit/tree/main/plugins/developer-kit-java/skills/langchain4j-testing-strategies

SKILL.md

LangChain4J Testing Strategies

Overview

Patterns for unit testing with mocks, integration testing with Testcontainers, and end-to-end validation of RAG systems, AI Services, and tool execution.

When to Use

  • Unit testing AI services: When you need fast, isolated tests for services using LangChain4j AiServices
  • Integration testing LangChain4j components: When testing real ChatModel, EmbeddingModel, or RAG pipelines with Testcontainers
  • Mocking AI models: When you need deterministic responses without calling external APIs
  • Testing LLM-based Java applications: When validating RAG workflows, tool execution, or retrieval chains

Instructions

1. Unit Testing with Mocks

Use mock models for fast, isolated testing. See references/unit-testing.md.

java
ChatModel mockModel = mock(ChatModel.class);
when(mockModel.generate(any(String.class)))
    .thenReturn(Response.from(AiMessage.from("Mocked response")));

var service = AiServices.builder(AiService.class)
        .chatModel(mockModel)
        .build();

2. Configure Testing Dependencies

Setup Maven/Gradle dependencies. See references/testing-dependencies.md.

  • langchain4j-test - Guardrail assertions
  • testcontainers - Containerized testing
  • mockito - Mock external dependencies
  • assertj - Fluent assertions

3. Integration Testing with Testcontainers

Test with real services. See references/integration-testing.md.

java
@Testcontainers
class OllamaIntegrationTest {
    @Container
    static GenericContainer<?> ollama = new GenericContainer<>(
        DockerImageName.parse("ollama/ollama:0.5.4")
    ).withExposedPorts(11434);

    @Test
    void shouldGenerateResponse() {
        // Verify container is healthy
        assertTrue(ollama.isRunning());
        await().atMost(30, TimeUnit.SECONDS)
            .until(() -> ollama.getLogs().contains("API server listening"));

        ChatModel model = OllamaChatModel.builder()
                .baseUrl(ollama.getEndpoint())
                .build();

        // Verify model responds before running tests
        assertDoesNotThrow(() -> model.generate("ping"));

        String response = model.generate("Test query");
        assertNotNull(response);
    }
}

4. Advanced Features

Streaming, memory, error handling patterns in references/advanced-testing.md.

5. Testing Workflow

Follow the testing pyramid from references/workflow-patterns.md:

  • 70% Unit Tests: Fast, isolated with mocks
  • 20% Integration Tests: Real services with health checks
  • 10% End-to-End Tests: Complete workflows
70% Unit Tests ─ Mock ChatModel, guardrails, edge cases
20% Integration Tests ─ Testcontainers, vector stores, RAG
10% End-to-End Tests ─ Complete user journeys

Troubleshooting

  • Container fails to start: Check Docker daemon is running, verify image exists, increase timeout
  • Model not responding: Verify baseUrl is correct, check container logs, ensure model is loaded
  • Test timeout: Increase @Timeout duration for slow models, check container resource limits
  • Flaky tests: Add retry logic or health checks before assertions

Examples

Unit Test

java
@Test
void shouldProcessQueryWithMock() {
    ChatModel mockModel = mock(ChatModel.class);
    when(mockModel.generate(any(String.class)))
        .thenReturn(Response.from(AiMessage.from("Test response")));

    var service = AiServices.builder(AiService.class)
            .chatModel(mockModel)
            .build();

    String result = service.chat("What is Java?");
    assertEquals("Test response", result);
}

Integration Test with Testcontainers

java
@Testcontainers
class RAGIntegrationTest {
    @Container
    static GenericContainer<?> ollama = new GenericContainer<>(
        DockerImageName.parse("ollama/ollama:0.5.4")
    );

    @BeforeAll
    static void waitForContainerReady() {
        await().atMost(60, TimeUnit.SECONDS)
            .until(() -> ollama.getLogs().contains("API server listening"));
    }

    @Test
    void shouldCompleteRAGWorkflow() {
        assertTrue(ollama.isRunning());

        var chatModel = OllamaChatModel.builder()
                .baseUrl(ollama.getEndpoint())
                .build();

        var embeddingModel = OllamaEmbeddingModel.builder()
                .baseUrl(ollama.getEndpoint())
                .build();

        var store = new InMemoryEmbeddingStore<>();
        var retriever = EmbeddingStoreContentRetriever.builder()
                .chatModel(chatModel)
                .embeddingStore(store)
                .embeddingModel(embeddingModel)
                .build();

        var assistant = AiServices.builder(RagAssistant.class)
                .chatLanguageModel(chatModel)
                .contentRetriever(retriever)
                .build();

        String response = assistant.chat("What is Spring Boot?");
        assertNotNull(response);
        assertTrue(response.contains("Spring"));
    }
}

Best Practices

  • Use @BeforeEach/@AfterEach for test isolation
  • Never call real APIs in unit tests; use mocks
  • Include @Timeout for external service calls
  • Test both success and error handling scenarios
  • Validate response coherence and edge cases

Common Patterns

Mock Strategy

java
ChatModel mockModel = mock(ChatModel.class);
when(mockModel.generate(anyString())).thenReturn(Response.from(AiMessage.from("Mocked")));
when(mockModel.generate(eq("Hello"))).thenReturn(Response.from(AiMessage.from("Hi")));
when(mockModel.generate(contains("Java"))).thenReturn(Response.from(AiMessage.from("Java")));

Assertion Helpers

java
assertThat(response).isNotNull().isNotEmpty();
assertThat(response).containsAll(expectedKeywords);
assertThat(response).doesNotContain("error");

Reference Documentation

  • Testing Dependencies - Maven/Gradle configuration
  • Unit Testing - Mock models, guardrails
  • Integration Testing - Testcontainers, real services
  • Advanced Testing - Streaming, memory, error handling
  • Workflow Patterns - Test pyramid, best practices

Constraints and Warnings

  • AI responses are non-deterministic; use mocks for reliable unit tests
  • Avoid real API calls in tests to prevent costs and rate limiting
  • Integration tests require Docker; use container health checks
  • RAG tests need properly seeded embedding stores
  • Mock-based tests cannot guarantee actual LLM behavior; supplement with integration tests
  • Use test-specific configuration profiles; never affect production data

Recommended Agent Skills

Expand your agent's capabilities with these related and highly-rated skills.

giuseppe-trisciuoglio/developer-kit

aws-cli-beast

Provides advanced AWS CLI patterns for managing EC2, Lambda, S3, DynamoDB, RDS, VPC, IAM, and CloudWatch. Generates bulk operation scripts, automates cross-service workflows, validates security configurations, and executes JMESPath queries for complex filtering. Triggers on "aws cli help", "aws command line", "aws scripting", "aws automation", "aws batch operations", "aws bulk operations", "aws cli pagination", "aws multi-region", "aws profiles", "aws cli troubleshooting".

192 20
Explore
giuseppe-trisciuoglio/developer-kit

aws-cost-optimization

Provides structured AWS cost optimization guidance using five pillars (right-sizing, elasticity, pricing models, storage optimization, monitoring) and twelve actionable best practices with executable AWS CLI examples. Use when optimizing AWS costs, reviewing AWS spending, finding unused AWS resources, implementing FinOps practices, reducing EC2/EBS/S3 bills, configuring AWS Budgets, or performing AWS Well-Architected cost reviews.

192 20
Explore
giuseppe-trisciuoglio/developer-kit

aws-sam-bootstrap

Provides AWS SAM bootstrap patterns: generates `template.yaml` and `samconfig.toml` for new projects via `sam init`, creates SAM templates for existing Lambda/CloudFormation code migration, validates build/package/deploy workflows, and configures local testing with `sam local invoke`. Use when the user asks about SAM projects, `sam init`, `sam deploy`, serverless deployments, or needs to bootstrap/migrate Lambda functions with SAM templates.

192 20
Explore
giuseppe-trisciuoglio/developer-kit

aws-drawio-architecture-diagrams

Creates professional AWS architecture diagrams in draw.io XML format (.drawio files) using official AWS Architecture Icons (aws4 library). Use when the user asks for AWS diagrams, VPC layouts, multi-tier architectures, serverless designs, network topology, or draw.io exports involving Lambda, EC2, RDS, or other AWS services.

192 20
Explore
giuseppe-trisciuoglio/developer-kit

aws-cloudformation-bedrock

Provides AWS CloudFormation patterns for Amazon Bedrock resources including agents, knowledge bases, data sources, guardrails, prompts, flows, and inference profiles. Use when creating Bedrock agents with action groups, implementing RAG with knowledge bases, configuring vector stores, setting up content moderation guardrails, managing prompts, orchestrating workflows with flows, and configuring inference profiles for model optimization.

192 20
Explore
giuseppe-trisciuoglio/developer-kit

aws-cloudformation-s3

Provides AWS CloudFormation patterns for Amazon S3. Use when creating S3 buckets, policies, versioning, lifecycle rules, and implementing template structure with Parameters, Outputs, Mappings, Conditions, and cross-stack references.

192 20
Explore

Didn't find tool you were looking for?

Be as detailed as possible for better results